Underwater swimming fish body mass estimation method and system

Abstract

The invention discloses an underwater swimming fish body mass estimation method and system, and relates to the technical field of computer vision, and the method comprises the steps: collecting fish body side view image data through a binocular camera, carrying out the calibration and correction operation, and carrying out the image preprocessing; the method comprises the following steps: extracting two-dimensional features of a fish body target, obtaining a final fish body image disparity map based on an APT-DMR method, extracting corresponding key matching points of a left image and a right image of a fish body based on the final disparity map, obtaining feature point space coordinates according to a triangular geometry principle, and calculating three-dimensional features by using a spatial distance formula; and constructing a fish body quality prediction model based on the MLR, and predicting the fish body quality. According to the method, the fish disparity map is obtained based on the SGBM algorithm, the corresponding key matching points of the left and right images of the fish body are extracted, the coordinates of the three-dimensional space feature points of the fish body are calculated, the three-dimensional features of the fish body target are extracted, the fish body quality prediction model based on the MLR is constructed, the fish body quality is predicted, and the prediction precision and reliability are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a method and system for estimating the mass of a fish swimming underwater. Background Art

[0002] Mass estimation refers to estimating the mass or mass distribution of objects in the target area in order to better understand the characteristics and conditions of the target area. Mass estimation is widely used in many fields. Accurately estimating the mass of fish is crucial for judging the growth status of fish, promoting accurate feeding in aquaculture, and improving the efficiency of aquaculture. Traditional methods often require fishing, weighing, and sampling. Such methods are not only time-consuming and labor-intensive, but may also cause stress and harm to fish. Therefore, a non-invasive mass measurement method is needed to efficiently and accurately measure the mass of fish swimming in water. With the rapid development of computer vision technology, a fish mass estimation method based on computer vision has emerged. Because of its advantages of objectivity, non-destructiveness, and ease of use, it has become an important means to study fish mass estimation methods. The fish mass measurement method based on computer vision first preprocesses the collected fish images to obtain the fish targets, then extracts the features of the fish images, calculates the eigenvalues, and finally fits the eigenvalues ​​of the fish images with the fish mass to establish a quality prediction model to achieve fish mass estimation.

[0003] In current aquaculture, invasive measurement operations can cause certain damage to fish bodies, affecting their growth and health; non-invasive measurements mostly use stereo cameras to capture fish images and extract two-dimensional and three-dimensional features to estimate quality. The calculation of three-dimensional features of fish bodies depends on the acquisition of accurate disparity maps. Currently, disparity maps are mostly acquired through manual participation to continuously debug algorithm parameters to obtain accurate disparity maps, which not only increases manpower and time costs, but also has a certain degree of randomness. In the process of studying the quality estimation method of underwater swimming fish, how to effectively obtain the disparity map of fish images, how to accurately obtain the features of fish images, and realize accurate estimation of the quality of underwater swimming fish is an urgent problem to be solved. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: the existing fish mass estimation method requires fishing, weighing and sampling, which is time-consuming and labor-intensive, causes stress and harm to the fish, and the invasive measurement operation affects the health of the fish; and the problem of how to non-invasively, efficiently and accurately measure the mass of fish swimming in water.

[0006] To solve the above technical problems, the present invention provides the following technical solutions: a method for estimating the mass of fish swimming underwater, comprising collecting side view image data of fish through a binocular camera and performing calibration and correction operations to perform image preprocessing; extracting two-dimensional features of the fish target, obtaining a final fish image disparity map based on the APT-DMR method, extracting corresponding key matching points of the left and right images of the fish based on the final disparity map, obtaining the spatial coordinates of the feature points according to the principle of triangular geometry, and calculating the three-dimensional features using the spatial distance formula; constructing a fish mass prediction model based on MLR to predict the fish mass.

[0007] As a preferred solution of the method for estimating the mass of fish swimming underwater described in the present invention, the calibration and correction operations include using a binocular camera to collect video data of fish swimming, capturing video frames to obtain fish images, calibrating the binocular camera to obtain camera parameters, and correcting the fish images based on the camera parameters to eliminate image distortion.

[0008] As a preferred solution of the method for estimating the mass of a swimming fish in the water of the present invention, the image preprocessing includes extracting the fish body ROI area, performing image processing operations of segmenting, binarizing and denoising the fish body image, and detecting and filtering the fish body contour; segmenting the fish body image includes segmenting the fish body foreground and background from the fish body image, which is expressed as:

[0009]

[0010] Among them, c represents the segmentation result, c i represents the label of the i-th pixel, Ω represents the fish body ROI area, and D i (c i ) is a data item used to measure the degree of match between the color of pixel i and the foreground and background models of the fish body, S ij (c i ,c j ) is a smoothing term used to measure the consistency between adjacent pixels, V i (c i ) represents the prior probability of the foreground and background of the fish body; binarization of the fish body image includes converting the grayscale fish body image into an image containing black and white, which is expressed as:

[0011]

[0012] Among them, Input(x,y) is the gray value of the pixel (x,y) in the original image, Output(x,y) is the value of the corresponding pixel in the binarized image, and threshold is a parameter used to determine the binarization threshold; the denoising of the fish image includes processing the segmented fish target through morphological closing operation, using a 5×5 kernel with all 1s to perform three closing operations to remove noise points in the image and fill holes, and adjust the connectivity and smoothness of the image. The closing operation process is expressed as:

[0013]

[0014] Among them, A is the input image, i.e. the fish body image, and B is the structural element. represents the expansion operation, Represents the corrosion operation; detecting and filtering the fish body contour includes extracting the fish body contour, including using the Canny algorithm to perform contour detection and filtering, and extracting the fish body contour, which is expressed as:

[0015]

[0016] θ(x,y)=arctan2(G y (x,y),G x (x,y) *

[0017] Among them, M(x,y) is the gradient amplitude of the image at the pixel point (x,y), G x (x,y) and G y (x, y) are the horizontal and vertical gradients at the pixel point (x, y), respectively, and θ(x, y) is the gradient direction of the image at the pixel point (x, y).

[0018] As a preferred solution of the method for estimating the mass of underwater swimming fish described in the present invention, the extraction of two-dimensional features of the fish target includes performing two-dimensional feature detection on the fish image based on the fish contour, using a minimum rectangular frame detection method for each fish image, and obtaining the two-dimensional features of the length, width and area of ​​the fish image through OpenCv functions.

[0019] As a preferred solution of the method for estimating the mass of a swimming fish underwater according to the present invention, the calculation of the three-dimensional features includes obtaining an initial disparity map by stereo matching based on the APT-DMR method and repairing the initial disparity map, and the stereo matching includes calculating the overall cost, which is expressed as:

[0020] C p (d)=P1·min(|I1(p)-I2(pd)|,P2)+P2·|dd prev |

[0021] Among them, C p (d) is the cost of the disparity value d at pixel p, I1(p) is the grayscale value of pixel p in the left image, I2(pd) is the grayscale value of the pixel in the right image that matches pixel p in the left image, and d prev is the disparity value of the previous pixel, P1 is the first parameter in the cost aggregation function, and P2 is the second parameter in the cost aggregation function; the main parameters involved in stereo matching include minimum disparity, disparity range, and matching block. Based on the idea of ​​Bayesian optimization algorithm, the main parameters are adapted by finding the optimal solution of the objective function to find the best parameter combination; a Gaussian process model is constructed to model the objective function f(x), and sampling is performed in the potential function space, which is expressed as:

[0022] f(x)~GP(m(x),k(x,x'))

[0023] Among them, m(x) is the mean function, k(x,x') is the covariance function, and in each iteration, the next candidate point is selected based on the confidence interval of the Gaussian process, expressed as:

[0024] x next =argmax(-μ(x)+βσ(x))

[0025] Among them, x next is the next candidate point, μ(x) is the mean predicted by the Gaussian process at position x, σ(x) is the variance predicted by the Gaussian process at position x, β is a hyperparameter that controls exploration and utilization, and the value of the generated objective function is used as the criterion for judging the quality of the initial disparity map. The value of the objective function is negatively correlated with the accuracy of the initial disparity map. The initial disparity map repair includes weighted disparity repair based on the KD tree, constructing a KD tree, and performing a neighbor search to find the index and distance of the nearest non-zero pixel of the abnormal value. Based on the index of the nearest neighbor point found, the normal value corresponding to the index of the nearest neighbor point is obtained, the weight and weighted sum are calculated, and the calculated weighted average value is filled back to the original abnormal pixel position. The neighbor search in the process is expressed as:

[0026]

[0027] Among them, p i' and q i' Respectively represent the coordinates of two data points in the i'th dimension, k is the dimension of the data point, the nearest neighbor is determined by the calculated distance, and the weighted average is expressed as:

[0028]

[0029] Among them, weights i”is the weight of the "i"th data point, weighted_sum is the weighted sum, weighted_average is the weighted average, d i” is the distance between the target and the ith data point, x i” is the value of the “i”th data point, and n is the total number of data points. The final disparity map is obtained by patching the initial disparity map obtained after stereo matching, and the corresponding key matching points of the left and right images of the fish body are extracted based on the final disparity map.

[0030] As a preferred solution of the method for estimating the mass of a fish swimming underwater according to the present invention, the calculation of the three-dimensional features further includes calculating the three-dimensional spatial coordinates of the fish feature points based on the final disparity map, and the x and y coordinates of the fish feature points in the three-dimensional space are expressed as:

[0031]

[0032] Among them, B is the baseline, which represents the distance between the two center points of the binocular camera, and X l and X r is the horizontal coordinate value of the target point on the left and right camera imaging planes, Y is the vertical coordinate value of the target point on the left and right camera imaging planes, f is the focal length of the camera, and the fish body depth information is the z-axis coordinate value in the three-dimensional spatial coordinates x, y, and z of the fish body feature information. The calculated depth value z is expressed as:

[0033]

[0034] The final disparity map is used to select the ratio of the maximum z-axis value and the minimum z-axis value in the fish body contour range as the three-dimensional feature of the fish body to predict the fish body quality. The fish body depth ratio depth_ratio is expressed as:

[0035]

[0036] Among them, Z max is the maximum depth of the fish body, Z min is the minimum depth value. After obtaining the coordinates of the feature points in three-dimensional space, the length and width feature values ​​are calculated using the Euclidean distance formula, expressed as:

[0037]

[0038] Among them, x1, y1, z1 are the coordinates of the first feature point of the fish body in the three-dimensional space, x2, y2, z2 are the coordinates of the second feature point of the fish body in the three-dimensional space, and value represents the three-dimensional length or width value of the fish body.

[0039] As a preferred solution of the method for estimating the mass of a fish swimming underwater according to the present invention, the predicted mass of the fish includes predicting the mass of the fish by establishing a multivariate linear regression model based on the two-dimensional characteristics and three-dimensional characteristics of the fish, which is expressed as:

[0040]

[0041] in, is the dependent variable, indicating the predicted fish mass, X1, X2, ..., X K is the independent variable, representing the characteristics of the fish body, β0, β1, β2, ..., β K is the regression coefficient, ε is the error term, and the model minimizes the residual sum of squares between the predicted value and the actual observed value by fitting the optimal regression coefficient, which is expressed as:

[0042]

[0043] in, is the estimated value of the regression coefficient, X is the matrix containing all independent variables, and represents the fish body feature matrix. is the vector of dependent variables, which represents the predicted fish mass vector; the residual term represents the difference between the model predicted value and the actual observed value, and the residual term ∈ is calculated as:

[0044]

[0045] Among them, Y' is the vector of actual observation values, which represents the true value vector of fish mass. ' is the vector of model prediction values, representing the vector of fish mass prediction values.

[0046] Another object of the present invention is to provide a system for estimating the mass of underwater swimming fish, which can extract the two-dimensional features of the fish target, obtain the final fish image disparity map based on the APT-DMR method, extract the corresponding key matching points of the left and right images of the fish based on the final disparity map, obtain the spatial coordinates of the feature points according to the principle of triangular geometry, and calculate the three-dimensional features using the spatial distance formula, thereby solving the problem of low efficiency in the current fish mass estimation.

[0047] As a preferred solution of the underwater swimming fish mass estimation system described in the present invention, it includes: a data acquisition and processing module, an image feature extraction module, and a quality prediction module; the data acquisition and processing module is used to collect fish side view image data through a binocular camera and perform calibration and correction operations to perform image preprocessing; the image feature extraction module is used to extract two-dimensional features of the fish target, obtain the final fish image disparity map based on the APT-DMR method, extract corresponding key matching points of the left and right images of the fish based on the final disparity map, obtain the spatial coordinates of the feature points according to the triangular geometry principle, and calculate the three-dimensional features using the spatial distance formula; the quality prediction module is used to construct a fish quality prediction model based on MLR to predict the fish quality.

[0048] A computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of a method for estimating the mass of a fish swimming underwater.

[0049] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of a method for estimating the mass of a fish swimming underwater.

[0050] Beneficial effects of the present invention: The method for estimating the mass of a swimming fish underwater provided by the present invention collects the side view image data of the fish through a binocular camera and performs correction operations, uses image processing technology to segment the corrected image to obtain the fish target, and extracts the two-dimensional features of the fish target. The two-dimensional features of the fish can be obtained in a non-invasive manner, avoiding the pressure and damage caused by fishing and weighing in traditional methods, and improving the efficiency and accuracy of data collection; an APT-DMR method is proposed to obtain the final fish image disparity map, which solves the problem of needing to constantly manually debug the algorithm parameter combination to accurately obtain the fish image disparity map; the corresponding key matching points of the left and right images of the fish are extracted. , calculate the coordinates of the three-dimensional spatial feature points of the fish body, and extract the three-dimensional features of the fish body target. The three-dimensional features of the fish body can be obtained in a non-invasive manner, avoiding the errors and inconveniences caused by measurement in traditional methods, and improving the integrity and accuracy of the data; a new fish body feature, the fish body depth ratio, is proposed, which enriches the feature representation of the model and improves the accuracy of fish body quality prediction; a fish body quality prediction model based on MLR is constructed to predict the fish body quality. The fish body quality can be predicted efficiently and accurately, avoiding the cumbersome operations and errors in traditional methods, and improving the accuracy and reliability of the prediction. The present invention achieves better results in terms of efficiency, accuracy and integrity. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0052] Figure 1 An overall flow chart of a method for estimating the mass of a fish swimming underwater provided in the first embodiment of the present invention.

[0053] Figure 2 A schematic diagram of fish body depth characteristics of a method for estimating the mass of a fish swimming underwater provided in a second embodiment of the present invention.

[0054] Figure 3 An overall flow chart of a system for estimating the mass of an underwater swimming fish provided in the third embodiment of the present invention. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0056] Example 1, reference Figure 1 , which is an embodiment of the present invention, provides a method for estimating the mass of a fish swimming underwater, comprising:

[0057] S1: The fish body side view image data is collected through the binocular camera and calibrated and corrected to perform image preprocessing.

[0058] Furthermore, the calibration and correction operations include using a binocular camera to collect video data of fish swimming, performing video frame capture to obtain fish images, calibrating the binocular camera to obtain camera parameters, and correcting the fish images based on the camera parameters to eliminate image distortion.

[0059] It should be noted that the image preprocessing includes extracting the fish body ROI area, performing image processing operations of segmenting, binarizing and denoising the fish body image, and detecting and filtering the fish body contour; segmenting the fish body image includes segmenting the fish body foreground and background from the fish body image, which is expressed as:

[0060]

[0061] Among them, C represents the segmentation result, C irepresents the label of the i-th pixel, Ω represents the fish body ROI area, and D i (c i ) is a data item used to measure the degree of match between the color of pixel i and the foreground and background models of the fish body, S ij (c i ,c j ) is a smoothing term used to measure the consistency between adjacent pixels, V i (c i ) represents the prior probability of the foreground and background of the fish body; binarization of the fish body image includes converting the grayscale fish body image into an image containing black and white, which is expressed as:

[0062]

[0063] Among them, Input(x,y) is the gray value of the pixel (x,u) in the original image, Output(x,y) is the value of the corresponding pixel in the binarized image, and threshold is a parameter used to determine the binarization threshold; the denoising of the fish image includes processing the segmented fish target through morphological closing operation, using a 5×5 kernel with all 1s to perform three closing operations to remove noise points in the image and fill holes, and adjust the connectivity and smoothness of the image. The closing operation process is expressed as:

[0064]

[0065] Among them, A is the input image, i.e. the fish body image, and B is the structural element. represents the expansion operation, Represents the corrosion operation; detecting and filtering the fish body contour includes extracting the fish body contour, including using the Canny algorithm to perform contour detection and filtering, and extracting the fish body contour, which is expressed as:

[0066]

[0067] θ(x,y)=arctan2(Gy(x,y),G x (x,y) *

[0068] Among them, M(x,y) is the gradient amplitude of the image at the pixel point (x,y), G x (x,y) and G y (x, y) are the horizontal and vertical gradients at the pixel point (x, y), respectively, and θ(x, y) is the gradient direction of the image at the pixel point (x, y).

[0069] It should also be noted that image preprocessing operations such as segmentation, binarization and noise reduction are intended to improve image quality and obtain better input data; two-dimensional features are detected and obtained using functions. These two-dimensional features provide an accurate data basis for subsequent three-dimensional feature extraction, simplify the data acquisition process, avoid the tedious operations of fish catching and weighing in traditional methods, and reduce the pressure and damage to the fish.

[0070] S2: Extract the two-dimensional features of the fish target, obtain the final fish image disparity map based on the APT-DMR method, extract the corresponding key matching points of the left and right images of the fish based on the final disparity map, obtain the spatial coordinates of the feature points according to the principle of triangular geometry, and calculate the three-dimensional features using the spatial distance formula.

[0071] Furthermore, extracting the two-dimensional features of the fish target includes performing two-dimensional feature detection on the fish image based on the fish contour and using a minimum rectangular frame detection method for each fish image, and obtaining the two-dimensional features of the length, width, and area of ​​the fish image through OpenCv functions.

[0072] It should be noted that calculating the three-dimensional features includes performing stereo matching based on the APT-DMR method to obtain an initial disparity map and repairing the initial disparity map. The stereo matching includes calculating the overall cost, which is expressed as:

[0073] C p (d)=P1·min(|I1(p)-I2(pd)|,P2)+P2·|dd prev |

[0074] Among them, C p (d) is the cost of the disparity value d at pixel p, I1(p) is the grayscale value of pixel p in the left image, I2(pd) is the grayscale value of the pixel in the right image that matches pixel p in the left image, and d prev is the disparity value of the previous pixel, P1 is the first parameter in the cost aggregation function, and P2 is the second parameter in the cost aggregation function; the main parameters involved in stereo matching include minimum disparity, disparity range, and matching block. Based on the idea of ​​Bayesian optimization algorithm, the main parameters are adapted by finding the optimal solution of the objective function to find the best parameter combination; a Gaussian process model is constructed to model the objective function f(x), and sampling is performed in the potential function space, which is expressed as:

[0075] f(x)~GP(m(x),k(x,x'))

[0076] Among them, m(x) is the mean function, k(x,x') is the covariance function, and in each iteration, the next candidate point is selected based on the confidence interval of the Gaussian process, expressed as:

[0077] xnext =argmax(-μ(x)+βσ(x))

[0078] Among them, x next is the next candidate point, μ(x) is the mean predicted by the Gaussian process at position x, σ(x) is the variance predicted by the Gaussian process at position x, β is a hyperparameter that controls exploration and utilization, and the value of the generated objective function is used as the criterion for judging the quality of the initial disparity map. The value of the objective function is negatively correlated with the accuracy of the initial disparity map. The initial disparity map repair includes weighted disparity repair based on the KD tree, constructing a KD tree, and performing a neighbor search to find the index and distance of the nearest non-zero pixel of the abnormal value. Based on the index of the nearest neighbor point found, the normal value corresponding to the index of the nearest neighbor point is obtained, the weight and weighted sum are calculated, and the calculated weighted average value is filled back to the original abnormal pixel position. The neighbor search in the process is expressed as:

[0079]

[0080] Among them, p i' and q i' Respectively represent the coordinates of two data points in the i'th dimension, k is the dimension of the data point, the nearest neighbor is determined by the calculated distance, and the weighted average is expressed as:

[0081]

[0082] Among them, weights i” is the weight of the "i"th data point, weighted_sum is the weighted sum, weighted_average is the weighted average, d i” is the distance between the target and the ith data point, x i” is the value of the “i”th data point, and n is the total number of data points. The final disparity map is obtained by patching the initial disparity map obtained after stereo matching, and the corresponding key matching points of the left and right images of the fish body are extracted based on the final disparity map.

[0083] It should also be noted that calculating the three-dimensional features also includes calculating the three-dimensional spatial coordinates of the fish feature points based on the final disparity map. The x and y coordinates of the fish feature points in the three-dimensional space are expressed as:

[0084]

[0085] Among them, B is the baseline, which represents the distance between the two center points of the binocular camera, and X l and X ris the horizontal coordinate value of the target point on the left and right camera imaging planes, Y is the vertical coordinate value of the target point on the left and right camera imaging planes, f is the focal length of the camera, and the fish body depth information is the z-axis coordinate value in the three-dimensional spatial coordinates x, y, and z of the fish body feature information. The calculated depth value z is expressed as:

[0086]

[0087] The final disparity map is used to select the ratio of the maximum z-axis value and the minimum z-axis value in the fish body contour range as the three-dimensional feature of the fish body to predict the fish body quality. The fish body depth ratio depth_ratio is expressed as:

[0088]

[0089] Among them, Z max is the maximum depth of the fish body, Z min is the minimum depth value. After obtaining the coordinates of the feature points in three-dimensional space, the length and width feature values ​​are calculated using the Euclidean distance formula, expressed as:

[0090]

[0091] Among them, x1, y1, z1 are the coordinates of the first feature point of the fish body in the three-dimensional space, x2, y2, z2 are the coordinates of the second feature point of the fish body in the three-dimensional space, and value represents the three-dimensional length or width value of the fish body.

[0092] It should also be noted that APT-DMR stands for Adaptive Parameter Tuning-Disparity MapRestoration, i.e. adaptive parameter adjustment-disparity map repair, solves the problem of needing to manually debug algorithm parameters to obtain accurate disparity maps; the fish depth ratio feature proposed in the present invention also enriches the feature representation of the model and improves the accuracy of fish quality prediction; the SGBM algorithm is used to accurately calculate the disparity map by optimizing the cost function and the Gaussian process model, and the corresponding key matching points of the left and right images of the fish body are extracted using the disparity map, and the coordinates of the three-dimensional feature points of the fish body are calculated by triangular geometry principles to obtain the three-dimensional information of the fish body (such as body length and body thickness), and the three-dimensional features of the fish body are obtained by non-invasive methods, providing more comprehensive fish body information, making up for the shortcomings of two-dimensional features, and improving the integrity and accuracy of the data. APT-DMR is an innovative method proposed by the present invention to obtain accurate disparity maps, wherein the APT part of the present invention uses SGBM to obtain the disparity map, and the adjustment of the SGBM algorithm parameters is adaptive under the APT method of the present invention; the DMR part of the present invention uses the KD tree to repair the disparity map, so that the obtained disparity map is more accurate. .

[0093] S3: Construct a fish quality prediction model based on MLR to predict fish quality.

[0094] Furthermore, the prediction of fish quality includes predicting the fish quality by establishing a multivariate linear regression model based on the two-dimensional and three-dimensional characteristics of the fish body, which is expressed as:

[0095]

[0096] in, is the dependent variable, indicating the predicted fish mass, X1, X2, ..., X k is the independent variable, representing the characteristics of the fish body, β0, β1, β2, ..., β K is the regression coefficient, ε is the error term, and the model minimizes the residual sum of squares between the predicted value and the actual observed value by fitting the optimal regression coefficient, which is expressed as:

[0097]

[0098] in, is the estimated value of the regression coefficient, X is the matrix containing all independent variables, and represents the fish body feature matrix. is the vector of dependent variables, which represents the predicted fish mass vector; the residual term represents the difference between the model predicted value and the actual observed value, and the residual term ∈ is calculated as:

[0099]

[0100] Among them, Y' is the vector of actual observation values, which represents the true value vector of fish mass. is the vector of model prediction values, representing the vector of fish mass prediction values.

[0101] It should be noted that MLR stands for Multiple Linear Regression, which is a regression algorithm. The present invention proposes a fish quality estimation model based on APT-DMR-MLR, which achieves good quality estimation effect. Based on two-dimensional and three-dimensional features, a multiple linear regression model is constructed to predict fish quality by fitting the relationship between features and quality. The model minimizes the residual sum of squares, optimizes the regression coefficient, ensures the accuracy of the prediction, and achieves accurate prediction of fish quality. It provides an efficient quality estimation method, supports accurate feeding and management in aquaculture, and improves the breeding efficiency.

[0102] Example 2, reference Figure 2 , which is an embodiment of the present invention, provides a method for estimating the mass of fish swimming underwater. In order to verify the beneficial effects of the present invention, scientific demonstration is carried out through economic benefit calculation and simulation experiments.

[0103] Firstly, a crucian carp image acquisition platform was built in the laboratory to acquire fish images. A binocular camera was used to collect fish swimming video data, and video frames were captured to obtain fish images. The binocular camera was calibrated to obtain camera parameters. The fish image was corrected based on the camera parameters to eliminate image distortion. The platform includes the following hardware equipment: a binocular camera (model ZED 2), an electronic scale (accuracy 0.1g), a fish breeding tank (90cm×65cm×70cm), a chopping board, and a measuring ruler (range 30cm, accuracy 1mm). The binocular camera was fixed about 30cm in front of the fish breeding tank, and the camera was connected to the computer through a network cable. The collected fish swimming video data was transmitted to the computer for storage, and video frames were captured from the collected fish swimming video data to obtain the fish image. In order to obtain the accurate parameters of the binocular camera when shooting fish swimming in water, the binocular camera was calibrated. Therefore, the calibration data was collected using a calibration board. The size of the main area of ​​the calibration plate is 200mm×200mm, the grid array is 12×9, and the size of each grid is 15mm×15mm. The calibration plate is inserted vertically into the water and swung slightly. A binocular camera is used to capture 20 sets of calibration plate images in different postures. The resolution of the collected left and right calibration images is 2208 pixels×1242 pixels. The second step is image preprocessing. Image preprocessing is a process of performing a series of processing steps on the original image in image analysis, computer vision and image processing tasks, aiming to improve the quality of the image, reduce noise, highlight features, and provide better input data for subsequent tasks. Fish images collected underwater are often There will be many problems such as uneven lighting, distortion caused by water waves and blur, which will lead to inaccurate feature extraction of fish images and affect the effect of fish quality estimation. Therefore, the fish image is preprocessed by ROI area extraction, image segmentation, image binarization, morphological operation noise reduction, contour detection and filtering. First, the region of interest of the fish is extracted and the extracted fish area is segmented. Image segmentation refers to the process of dividing a digital image into multiple sub-regions or pixel sets, so that each sub-region has specific attributes or characteristics, so as to understand the image content and extract the target. The GrabCut algorithm is an iterative optimization-based Image segmentation technology combines the initial mark and color information provided by the user, and continuously adjusts the foreground and background probabilities of each pixel to minimize the energy function to achieve automatic image segmentation. The segmentation results are accurate and reliable. Therefore, the GrabCut algorithm is used to segment the fish body from the fish body image. Secondly, the fish body image is binarized. Image binarization is the process of converting a grayscale image into an image containing only two colors (usually black and white). It can help extract specific features and details of the target object in the image, making the target more obvious and prominent in the image, which is convenient for target detection and segmentation. Then, the fish body image is subjected to morphological operations for noise reduction.Morphological operation refers to the process of performing mathematical morphological operations on a set of pixel points in an image to achieve the purpose of image denoising, etc. The segmented fish body target is processed through morphological closing operations, and a 5×5 kernel with all 1s is used to perform three closing operations to remove noise points in the image and fill holes to improve the connectivity and smoothness of the image, thereby improving the quality and accuracy of the image segmentation results. Finally, the fish body image is detected and filtered. Since the boundary of the fish body target obtained by segmentation is similar to the background color or texture, the boundary of the object cannot be accurately captured, and the detected fish body target may have some noise, discontinuous small line segments or irrelevant details. On this basis, the Canny algorithm is used for contour detection and Filtering is performed to achieve accurate extraction of the fish body contour, which provides a more reliable guarantee for subsequent processing such as extraction of image features. The third step is to extract the two-dimensional features of the fish body. Image feature extraction is the process of extracting relevant information describing the image content, structure or attributes from the image. By analyzing the two-dimensional features of the fish body's external morphology, size, proportion, etc., it is possible to find the laws and trends related to the quality of the fish body, which is of great help in predicting the quality of the fish body. Based on the detected fish body contour, the OpenCv library function is used to obtain its minimum external rectangular frame, and on this basis, the two-dimensional features such as length, width, and area are extracted. The two-dimensional features of the fish body mentioned in our invention are all in pixels, and a total of 7 two-dimensional features are extracted. Refer to Table 1 to record the two-dimensional features of the fish body image. ,

[0104] Table 1 List of two-dimensional features of fish images

[0105]

[0106] The fourth step is to obtain the three-dimensional features of the fish image. Although the two-dimensional features can provide some important information about the appearance of the fish, they only provide the shape of the fish on a single plane. In fact, the fish itself is a three-dimensional structure, which not only has information such as body length and area on the plane, but also has depth information such as fish thickness. For example, even if two fish are the same in length and area, their qualities will be different due to their different body thicknesses. Therefore, in order to further describe the shape, size, structure and other information of the fish, the three-dimensional features of the fish, such as three-dimensional body length and three-dimensional body width, are also extracted. On this basis, the three-dimensional features of the fish are extracted. The depth ratio features of the fish body are taken to realize the three-dimensional cognition of the fish body. Based on the extracted two-dimensional features of the fish body, stereo matching is first performed based on the proposed APT-DMR method to obtain the disparity map of the fish image. Secondly, the feature points of the left and right binocular images are matched. Then, the coordinates of these feature points in the three-dimensional space are calculated through the triangular geometry principle. Finally, the final three-dimensional feature value of the fish body is obtained through the spatial distance calculation formula, including disparity map acquisition, disparity map repair, and three-dimensional feature calculation. The fifth step is to construct a fish quality prediction model, and the prediction of fish quality is realized by establishing a multivariate linear regression model.

[0107] Example 3, reference Figure 3 , which is an embodiment of the present invention, provides a system for estimating the mass of a swimming fish underwater, including a data acquisition and processing module, an image feature extraction module, and a mass prediction module.

[0108] The data acquisition and processing module is used to collect the side view image data of the fish body through the binocular camera and perform calibration and correction operations to perform image preprocessing; the image feature extraction module is used to extract the two-dimensional features of the fish body target, obtain the final fish body image disparity map based on the APT-DMR method, extract the corresponding key matching points of the left and right images of the fish body based on the final disparity map, obtain the spatial coordinates of the feature points according to the principle of triangular geometry, and calculate the three-dimensional features using the spatial distance formula; the quality prediction module is used to construct a fish body quality prediction model based on MLR to predict the fish body quality.

[0109] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0110] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0111] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0112] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc. It should be noted that the above embodiments are only used to illustrate the technical solution of the present invention and are not limited. Although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

[0113] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for estimating the mass of a fish swimming underwater, characterized in that: include: The side view image data of the fish body is collected by using a binocular camera and calibrated and corrected to perform image preprocessing; Extract the two-dimensional features of the fish target, obtain the final fish image disparity map based on the APT-DMR method, extract the corresponding key matching points of the left and right images of the fish based on the final disparity map, obtain the spatial coordinates of the feature points according to the principle of triangular geometry, and calculate the three-dimensional features using the spatial distance formula; Construct a fish quality prediction model based on MLR to predict fish quality.

2. The method for estimating the mass of a fish swimming underwater as claimed in claim 1, characterized in that: The calibration and correction operation includes using a binocular camera to collect fish swimming video data, performing video frame capture to obtain a fish image, calibrating the binocular camera to obtain camera parameters, and correcting the fish image based on the camera parameters to eliminate image distortion.

3. The method for estimating the mass of a fish swimming underwater as claimed in claim 2, characterized in that: The image preprocessing includes extracting the fish body ROI area, performing image processing operations of segmenting, binarizing and denoising the fish body image, and detecting and filtering the fish body contour; Segmenting the fish image includes segmenting the fish foreground and background from the fish image, which can be expressed as: Among them, c represents the segmentation result, c i represents the label of the i-th pixel, Ω represents the fish body ROI area, and D i (c i ) is a data item used to measure the degree of match between the color of pixel i and the foreground and background models of the fish body, S ij (c i ,c j ) is a smoothing term used to measure the consistency between adjacent pixels, V i (c i ) represents the prior probability of the fish foreground and background; Binarization of the fish image includes converting the grayscale fish image into an image containing black and white, which is expressed as: Among them, Input(x,y) is the grayscale value of the pixel (x,u) in the original image, Output(x,y) is the value of the corresponding pixel in the binarized image, and threshold is a parameter used to determine the binarization threshold; The denoising of the fish image includes processing the segmented fish target through morphological closing operation, using a 5×5 kernel with all 1s to perform three closing operations to remove noise points in the image and fill holes, and adjust the connectivity and smoothness of the image. The closing operation process is expressed as: Among them, A is the input image, i.e. the fish body image, and B is the structural element. represents the expansion operation, Indicates corrosion operation; The detection and filtering of the fish body contour includes extracting the fish body contour, including using the Canny algorithm to perform contour detection and filtering, and extracting the fish body contour, which is expressed as: θ(x,y)=arctan2(G y (x,y), G x (x,y)) Among them, M(x,y) is the gradient amplitude of the image at the pixel point (x,y), G x (x,y) and G y (x, y) are the horizontal and vertical gradients at the pixel point (x, y), respectively, and θ(x, y) is the gradient direction of the image at the pixel point (x, y).

4. The method for estimating the mass of a fish swimming underwater as claimed in claim 3, characterized in that: The method for extracting the two-dimensional features of the fish target includes performing two-dimensional feature detection on the fish image based on the fish outline by using a minimum rectangular frame detection method for each fish image, and obtaining the two-dimensional features of the length, width and area of ​​the fish image through an OpenCv function.

5. The method for estimating the mass of a fish swimming underwater as claimed in claim 4, characterized in that: The calculation of the three-dimensional features includes performing stereo matching based on the APT-DMR method to obtain an initial disparity map and repairing the initial disparity map. The stereo matching includes calculating the overall cost, which is expressed as: C p (d)=P1·min(|I1(p)-I2(p-d)|,P2)+P2·|d-d prev | Among them, C p (d) is the cost of the disparity value d at pixel p, I1(p) is the grayscale value of pixel p in the left image, I2(pd) is the grayscale value of the pixel in the right image that matches pixel p in the left image, and d prev is the disparity value of the previous pixel, P1 is the first parameter in the cost aggregation function, and P2 is the second parameter in the cost aggregation function; The main parameters involved in stereo matching include minimum disparity, disparity range, and matching blocks. Based on the idea of ​​Bayesian optimization algorithm, the main parameters are adapted by finding the optimal solution of the objective function to find the best parameter combination. Construct a Gaussian process model, model the target function f(x), and sample in the potential function space, which can be expressed as: f(x)~GP(m(x),k(x,x')) Among them, m(x) is the mean function, k(x,x') is the covariance function, and in each iteration, the next candidate point is selected based on the confidence interval of the Gaussian process, expressed as: x next =argmax(-μ(x)+βσ(x)) Among them, x next is the next candidate point, μ(x) is the mean predicted by the Gaussian process at position x, σ(x) is the variance predicted by the Gaussian process at position x, β is a hyperparameter that controls exploration and utilization, and the value of the generated objective function is used as the criterion for judging the quality of the initial disparity map. The value of the objective function is negatively correlated with the accuracy of the initial disparity map. The inpainting of the initial disparity map includes weighted inpainting of the disparity based on the KD tree, constructing the KD tree, performing a neighbor search to find the index and distance of the nearest non-zero pixel of the abnormal value, obtaining the normal value corresponding to the index of the nearest neighbor based on the index of the nearest neighbor found, calculating the weight and weighted sum, and filling the calculated weighted average back to the original abnormal pixel position. The neighbor search in the process is expressed as: Among them, p i' and q i' Respectively represent the coordinates of two data points in the i'th dimension, k is the dimension of the data point, the nearest neighbor is determined by the calculated distance, and the weighted average is expressed as: Among them, weights i” is the weight of the "i"th data point, weighted_sum is the weighted sum, weighted_average is the weighted average, d i” is the distance between the target and the ith data point, x i” is the value of the ith data point, and n is the total number of data points; The final disparity map is obtained by patching the initial disparity map obtained after stereo matching, and the corresponding key matching points of the left and right images of the fish body are extracted based on the final disparity map.

6. The method for estimating the mass of a fish swimming underwater as claimed in claim 5, characterized in that: The calculating of the three-dimensional features further includes calculating the three-dimensional spatial coordinates of the fish feature points based on the final disparity map, and the x and y coordinates of the fish feature points in the three-dimensional space are expressed as: Among them, B is the baseline, which represents the distance between the two center points of the binocular camera, and X l and X r is the horizontal coordinate value of the target point on the left and right camera imaging planes, Y is the vertical coordinate value of the target point on the left and right camera imaging planes, f is the focal length of the camera, and the fish body depth information is the z-axis coordinate value in the three-dimensional spatial coordinates x, y, and z of the fish body feature information. The calculated depth value z is expressed as: The final disparity map is used to select the ratio of the maximum z-axis value and the minimum z-axis value in the fish body contour range as the three-dimensional feature of the fish body to predict the fish body quality. The fish body depth ratio depth_ratio is expressed as: Among them, Z max is the maximum depth of the fish body, Z min is the minimum depth value. After obtaining the coordinates of the feature points in three-dimensional space, the length and width feature values ​​are calculated using the Euclidean distance formula, expressed as: Among them, x1, y1, z1 are the coordinates of the first feature point of the fish body in the three-dimensional space, x2, y2, z2 are the coordinates of the second feature point of the fish body in the three-dimensional space, and value represents the three-dimensional length or width value of the fish body.

7. The method for estimating the mass of a fish swimming underwater as claimed in claim 6, characterized in that: The method of predicting the fish body quality includes predicting the fish body quality by establishing a multiple linear regression model based on the two-dimensional characteristics and three-dimensional characteristics of the fish body, which is expressed as: in, is the dependent variable, indicating the predicted fish mass, X1, X2, ..., X K is the independent variable, representing the characteristics of the fish body, β0, β1, β2, ..., β K is the regression coefficient, ε is the error term, and the model minimizes the residual sum of squares between the predicted value and the actual observed value by fitting the optimal regression coefficient, which is expressed as: in, is the estimated value of the regression coefficient, X is the matrix containing all independent variables, and represents the fish body feature matrix. is the vector of dependent variables, representing the predicted fish mass vector; The residual term represents the difference between the model prediction value and the actual observation value. The residual term ∈ is calculated as: Among them, Y' is the vector of actual observation values, which represents the true value vector of fish mass. is the vector of model prediction values, representing the vector of fish mass prediction values.

8. A system using the method for estimating the mass of a fish swimming underwater as claimed in any one of claims 1 to 7, characterized in that: It includes data acquisition and processing module, image feature extraction module and quality prediction module; The data acquisition and processing module is used to collect fish body side view image data through a binocular camera and perform calibration and correction operations to perform image preprocessing; The image feature extraction module is used to extract the two-dimensional features of the fish target, obtain the final fish image disparity map based on the APT-DMR method, extract the corresponding key matching points of the left and right images of the fish based on the final disparity map, obtain the spatial coordinates of the feature points according to the triangular geometry principle, and calculate the three-dimensional features using the spatial distance formula; The quality prediction module is used to construct a fish quality prediction model based on MLR to predict fish quality.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for estimating the mass of a fish swimming underwater according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for estimating the mass of a fish swimming underwater according to any one of claims 1 to 7 are implemented.

Images