A method and system for detecting underwater fish biomass

Abstract

The application discloses a kind of underwater fish biomass detection method and system, comprising: after underwater binocular image pair is divided by instance, target fish binocular image pair is obtained, and corresponding mask pair is extracted from target fish binocular image pair;Stereo matching is carried out to the mask pair to obtain disparity map, and the depth value in adaptive window is traversed to the disparity map and is carried out truncated gaussian mean filtering and missing depth value completion, until the number of depth values remaining after filtering in window meets the proportion requirement, to obtain dense disparity map;After disparity smoothing processing to dense disparity map, three-dimensional reconstruction is carried out, to obtain fish contour, fish feature points in fish contour are extracted and the distance between fish feature points is calculated, to obtain the fork length and body width of target fish, according to the correlation function of fork length, body width and weight, the weight of target fish is estimated. Improve the edge detection precision of complete target in binocular image, realize the estimation of multi-pose underwater fish non-contact biomass.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fish detection, in particular to a method and system for detecting underwater fish biomass. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] The existing underwater binocular vision method has strict requirements on the posture of the target fish when identifying and detecting the fish, and has high effectiveness for fish with an orthogonal camera optical axis and a straight central axis, but lacks robustness for fish with a non-orthogonal camera optical axis and a curved posture. The non-orthogonal camera optical axis of the fish refers to the fish facing the camera obliquely, such as the head being close to the camera and the tail being far away from the camera, which can be a straight posture or a curved posture. The curved posture refers to the spine of the fish being curved, in which case the profile of the fish is a curved surface. At present, this part of data with complete fish profile is often filtered out during identification and detection, and thus the problem of identifying fish with a non-orthogonal camera optical axis or a curved posture needs to be solved.

[0004] When detecting the biomass of the identified target fish, especially for body weight estimation, the current method either uses a single variable such as body length or circumference for linear fitting of the body weight, which has low precision, or directly applies existing formulas without considering the differences in body weight estimation of different types of fish, which is not universal. SUMMARY

[0005] To solve the above problems, the present application provides a method and system for detecting underwater fish biomass, which designs a truncated Gaussian mean filtering method with an adaptive window to obtain a dense disparity map, performs disparity smoothing on the dense disparity map, and then performs three-dimensional reconstruction to obtain the fish profile, thereby improving the target detection accuracy and realizing non-contact biomass estimation of underwater fish with multiple postures.

[0006] To achieve the above purpose, the present application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a method for detecting underwater fish biomass, comprising:

[0008] After the underwater binocular image pair is instance segmented, a target fish binocular image pair is obtained, and a corresponding mask pair is extracted from the target fish binocular image pair;

[0009] The stereo matching is performed on the mask pair to obtain a disparity map, the adaptive window is traversed for a depth value in the disparity map, and the truncated Gaussian mean filtering and missing depth value completion are performed until the number of remaining depth values in the window meets the proportion requirement, so as to obtain a dense disparity map;

[0010] The three-dimensional reconstruction is performed on the dense disparity map after the disparity smoothing processing, a fish body contour is obtained, fish body feature points in the fish body contour are extracted, and the distance between the fish body feature points is calculated, so as to obtain the fork length and body width of the target fish body, and the body weight of the target fish body is estimated according to the correlation function of the fork length, the body width and the body weight.

[0011] As an optional implementation, the process of instance segmentation on the underwater binocular image pair includes:

[0012] The multi-scale attention feature extraction is performed on the underwater binocular image pair, the triple attention extraction is performed on the attention feature map of each scale after the up-sampling, and the region candidate frame of the underwater binocular image pair is generated.

[0013] The target fish body is detected and segmented according to the region candidate frame, the mask is segmented according to the judgment on the pixel category in the region candidate frame, and the target fish body and the corresponding mask are extracted.

[0014] As an optional implementation, the process of the truncated Gaussian mean filtering includes: taking each pixel point in the disparity map as the center, traversing all the depth values in a window with an edge length of N depth values, fitting the non-zero depth values in the window into a Gaussian distribution, and filtering the depth values in the window with the twice standard deviation of the Gaussian distribution as the truncated mean value parameter.

[0015] As an optional implementation, the process of the missing depth value completion includes: replacing the depth value at the filtering point with the average of the remaining depth values after filtering, and if the number of the remaining depth values after filtering does not exceed the set threshold, the window size is expanded until the proportion of the remaining depth values meets the requirement, and the dense depth map is obtained.

[0016] As an optional implementation, the process of the three-dimensional reconstruction includes: converting the dense disparity map into a point cloud map, performing voxel down-sampling on the point cloud map, rotating the point cloud map to the target fish body axis parallel to the X axis and the contour perpendicular to the Z axis, and finally obtaining the fish body contour in the triangular mesh format through the Poisson surface reconstruction.

[0017] As an optional implementation, the fish body feature points of the head, the tail, the back and the abdomen of the fish body contour are extracted, the distances between all the feature points of the head and the tail, the back and the abdomen are calculated on the triangular mesh surface, so as to obtain the fork length and the body width of the target fish body.

[0018] As an alternative embodiment, the correlation function of fork length, body width and body weight is fitted by introducing a cost function as: W=0.00477507fl+0.0311315wl-4.09581, wherein fl is the fork length, wl is the body width, and W is the body weight.

[0019] In a second aspect, the present application provides an underwater fish biomass detection system, comprising:

[0020] An instance segmentation module is configured to perform instance segmentation on the underwater binocular image pair to obtain a target fish binocular image pair, and extract a corresponding mask pair from the target fish binocular image pair.

[0021] A disparity map processing module is configured to perform stereo matching on the mask pair to obtain a disparity map, and traverse the depth values in the disparity map using an adaptive window, and perform truncated Gaussian mean filtering and missing depth value completion until the number of remaining depth values in the window after filtering meets the proportion requirement, thereby obtaining a dense disparity map.

[0022] A biomass detection module is configured to perform three-dimensional reconstruction on the dense disparity map after disparity smoothing processing to obtain a fish contour, extract fish feature points in the fish contour and calculate the distance between the fish feature points, thereby obtaining the fork length and body width of the target fish, and estimating the body weight of the target fish according to the correlation function of the fork length, body width and body weight.

[0023] In a third aspect, the present application provides an electronic device comprising a memory and a processor, and computer instructions stored on the memory and running on the processor, wherein when the computer instructions are run by the processor, the method of the first aspect is completed.

[0024] In a fourth aspect, the present application provides a computer readable storage medium for storing computer instructions, wherein when the computer instructions are executed by a processor, the method of the first aspect is completed.

[0025] Compared with the prior art, the present application has the following beneficial effects:

[0026] The present application provides an underwater fish biomass detection method and system, which uses an instance segmentation network model and modifies its backbone network, increases the attention mechanism module to improve the attention ability of multi-scale channels to features, improve the segmentation accuracy, solve the problems of texture repetition and edge blur in binocular images, and accurately extract the target fish with complete contour.

[0027] The application provides an underwater fish biomass detection method and system, disparity maps are obtained through stereo matching, a truncated Gaussian mean filtering method of a self-adaptive window is designed to obtain a dense disparity map, three-dimensional reconstruction is performed on the dense disparity map after disparity smoothing, and thus fish body contours are obtained, the fish body contour and the camera optical axis are non-orthogonal or the fish body recognition problem of posture bending is solved, the edge detection precision of a complete target in binocular images is improved, and non-contact estimation of underwater fish biomass in multiple postures is realized.

[0028] The application provides an underwater fish biomass detection method and system, a cost function is introduced to fit prediction formulas of fork length, body width and body weight of a target fish, the influence of the body width on the body weight is considered, and the prediction formulas have universality.

[0029] Advantages of additional aspects of the application will be partially given in the following description, partially become obvious from the following description, or be known by practice of the application. BRIEF DESCRIPTION OF DRAWINGS

[0030] The drawings accompanying the specification of the application form part of the application and serve to provide further understanding of the application, the illustrative embodiments of the application and their description serve to explain the application and do not constitute an improper limitation of the application.

[0031] Figure 1 A flowchart of an underwater fish biomass detection method provided for the embodiment 1 of the application is shown in the figure;

[0032] Figure 2 A schematic diagram of an instance segmentation backbone network provided for the embodiment 1 of the application is shown in the figure;

[0033] Figure 3 A schematic diagram of a region proposal network and a prediction branch provided for the embodiment 1 of the application is shown in the figure;

[0034] Figure 4 A training result diagram of instance segmentation provided for the embodiment 1 of the application is shown in the figure;

[0035] Figure 5 A training loss diagram of an instance segmentation network provided for the embodiment 1 of the application is shown in the figure;

[0036] Figure 6 An example diagram of three-dimensional reconstruction results provided for the embodiment 1 of the application is shown in the figure;

[0037] Figure 7 A schematic diagram of fish feature point extraction provided for the embodiment 1 of the application is shown in the figure;

[0038] Figure 8 A schematic diagram of fork length estimation provided for the embodiment 1 of the application is shown in the figure;

[0039] Figure 9 A schematic diagram of body width estimation provided for the embodiment 1 of the application is shown in the figure. Detailed Implementation

[0040] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0041] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0042] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. Furthermore, it should be understood that the terms “comprising” and “including”, and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or apparatus.

[0043] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0044] Example 1

[0045] like Figure 1 As shown, this embodiment provides a method for detecting underwater fish biomass, including:

[0046] After instance segmentation of the underwater binocular image pairs, the target fish binocular image pairs are obtained, and the corresponding mask pairs are extracted from the target fish binocular image pairs.

[0047] A disparity map is obtained by stereo matching of the mask pairs. The disparity map is then subjected to adaptive window traversal of depth values, truncated Gaussian mean filtering, and missing depth values ​​are filled in until the number of remaining depth values ​​after filtering within the window meets the proportional requirements, thus obtaining a dense disparity map.

[0048] After parallax smoothing, the dense parallax map is reconstructed in three dimensions to obtain the fish body outline. The fish body feature points in the fish body outline are extracted and the distance between the fish body feature points is calculated. Thus, the fork length and body width of the target fish body are obtained. Based on the correlation function of fork length, body width and weight, the weight of the target fish body is estimated.

[0049] In this embodiment, the binocular camera is first calibrated and corrected;

[0050] Specifically, a world coordinate system is established in the real world with a unit of m; a camera coordinate system is established in the real world with the optical center of the binocular camera as the origin with a unit of m; an image coordinate system is established in the image with the principal point of the image as the origin with a unit of mm; a pixel coordinate system is established in the image with the upper left corner of the image as the origin with a unit of pixels;

[0051] The above four coordinate systems are related by Zhang's calibration method as follows:

[0052]

[0053] wherein, Z c is the vertical coordinate value of the point in the camera coordinate system; (u, v) is the coordinate of the pixel corresponding to the point in the pixel coordinate system; f x , f y are the ratio of the camera focal length to the physical size of the x-axis of the image and the ratio of the camera focal length to the physical size of the y-axis of the image, respectively; (u0, v0) is the principal point coordinate of the camera; R is the rotation matrix from the world coordinate system to the camera coordinate system, and T is the translation matrix from the world coordinate system to the camera coordinate system; (X w , Y w , Z w ) is the coordinate of the point in the world coordinate system.

[0054] The binocular camera is fixed underwater, and the checkerboard calibration plate is held by hand so that the calibration plate appears completely in the picture of the binocular camera and occupies an area of about 1 / 4 to 1 / 2 of the image; a plurality of groups of calibration plate images at different positions and angles are photographed by using the binocular camera, Zhang's calibration method is used, the double target calibration toolbox (Stereo Camera Calibrator) in Matlab is used to solve the internal and external parameters of the camera, the OpenCV program is used to extract the fish binocular image containing the contour without occlusion and the edge is clear, and the corrected fish binocular image is cropped to a preset size, such as 960x960 pixels.

[0055] The corrected and cropped fish binocular image is manually labeled by using a labeling software. When labeling, the dorsal fin and anal fin are not included. After labeling, a file in json format is generated, the file name is consistent with the image name, the labeled data is flipped left and right to realize the expansion of the data set, and finally the training set, the verification set and the test set are randomly divided according to the ratio of 8:1:1.

[0056] In view of the problems of texture repetition and edge blur existing in the collected binocular images, in order to obtain a target fish body with complete contour, the example segmentation model of the embodiment adopts a convolutional neural network algorithm of deep learning, and is based on an example segmentation network Mask R-CNN. A pyramid spatial attention (PSA) module is integrated in a backbone network ResNet-50. The convolutional kernel with an original size of 3*3 and a stride of 1 of a convolutional layer is replaced by the PSA module, so as to improve the attention ability of multi-scale channels to features.

[0057] The PSA module is connected with a feature pyramid network (FPN), and a feature map processed by the FPN is upsampled and then introduced into a lightweight triplet attention module, so as to capture cross-dimension features and improve the performance of the algorithm in an example segmentation task.

[0058] The feature map obtained after the operation of the triplet attention mechanism is subjected to region proposal network (RPN) to generate a region candidate box. After further screening, the region candidate box is introduced into a target detection branch and an example segmentation branch. In the target detection branch, the position, category and confidence of a prediction box are obtained. In the example segmentation branch, whether each pixel in the candidate box belongs to a target category is scored in terms of confidence.

[0059] Then, as shown in FIG. 1, the specific processing procedure of the example segmentation model includes: Figures 2-3

[0060] The binocular images in the training set are input into a ResNet50+PSA network structure with Resnet50 pre-training weights to perform multi-scale attention feature extraction, and four multi-scale attention feature maps with sizes of 200*200*256, 100*100*512, 50*50*1024 and 25*25*2048 are obtained.

[0061] The above feature maps are input into an FPN+Triplet Attention network, and triplet attention is extracted from each scale of the attention feature maps, so as to obtain five feature maps with different sizes and a channel number of 256, that is, 200*200, 100*100, 50*50, 25*25 and 13*13.

[0062] The binocular images and the corresponding feature maps are input into a region proposal network (RPN) to generate a region candidate box for each binocular image.

[0063] ​The feature map obtained in the FPN+Triplet Attention network and the region candidate box obtained by the RPN are input into the RoI Align, and the vertex coordinates of the region candidate box are mapped onto the feature maps of different sizes by a bilinear interpolation method; the ROI Align is a region feature aggregation method;

[0064] A target detection branch (i.e., a bounding box branch) and an instance segmentation branch (i.e., a mask branch) are introduced; in the target detection branch, the position, class and confidence of the bounding box are predicted according to the output of the RoI Align, and in the instance segmentation branch, whether each pixel in the bounding box belongs to the target class is scored for confidence, and the pixels with a confidence higher than a set threshold are extracted to form a target mask;

[0065] The bounding box, the predicted class, the class confidence and the target mask are drawn in the original image and then output, thereby extracting the target fish body and the corresponding mask.

[0066] Finally, the binocular image pairs in which the same target fish body appears in consecutive frames are selected from the corrected and cropped binocular images, and the left-eye image and the right-eye image at the same moment are referred to as one binocular image pair, and the corresponding mask pair is extracted from the target fish body binocular image pair, and the segmentation result is as shown in Figure 4 .

[0067] In the training phase, the target detection loss, the bounding box regression loss and the mask segmentation loss of the training set and the validation set are calculated to adjust the weight matrix in the neural network by the back propagation method, and the model parameters are constantly optimized by calculating the mAP, AP 75 and mIoU.

[0068] When training the neural network, the batch size of the input image is set to 16, the size of the image is normalized to 800x800 pixels, the initial learning rate of the model is 0.01, the gradual training strategy is used in the first round, the learning rate is divided by 10 at the 4th and 8th rounds, the training is performed for 20 rounds, the stochastic gradient descent algorithm (SGD) with momentum is selected as the model optimizer, the momentum factor is 0.9, and the weight decay parameter is 0.0001. The training loss is as shown in Figure 5 . The trained network model is verified on the test set to evaluate the indicators AP 50:95 , AP 75 and mIoU, and the results are AP 50:95 0.812, AP 75 0.968 and mIoU 0.796;

[0069]

[0070]

[0071]

[0072] where TP is the foreground pixel predicted as foreground, FP is the background pixel predicted as foreground, FN is the foreground pixel predicted as background, n is the number of pictures in the dataset, A i is the mask predicted by the model, B i is the true value of the mask.

[0073] In the embodiment, after the binocular image to be tested is subjected to instance segmentation by the above trained model, a mask pair is generated, a disparity map is obtained by using a stereo matching network SGBM on the mask pair, and depth values outside the mask are filtered; wherein, the stereo matching network is set with a penalty coefficient p1 of 225, a p2 of 900, a minimum search range of 256, and a disparity block size of 15*15;

[0074] Then, the adaptive window truncated Gaussian mean filtering method is used to perform truncated Gaussian mean filtering and missing depth value completion on the disparity map; specifically:

[0075] Each pixel point in the disparity map is taken as the center, and a search window with a side length of N depth values is used to traverse all depth values; the non-0 depth values in the search window are fitted as a Gaussian distribution, and the depth values in the search window are filtered with the twice standard deviation of the Gaussian distribution as the truncated mean parameter; the remaining depth values after filtering are averaged to replace the depth value at the filtering point; if the number of remaining depth values after filtering does not exceed the set threshold, for example, 1 / 2 of the non-0 depth values in the search window in the embodiment, the size of the search window is enlarged until the proportion of the remaining depth values reaches 1 / 2, and a dense depth map is obtained.

[0076] For the matching error result in the dense depth map, the least square fitting method is used to perform disparity smoothing processing on the dense disparity map; specifically:

[0077] The abscissa of each point in the dense depth map is taken as the independent variable, and the depth value is taken as the dependent variable:

[0078] A = {(i, d i,j ) | (i, j) e M}

[0079] where (i, j) is the coordinate of the point in the depth map, M is the mask, d i,j is the depth value obtained by the adaptive window truncated Gaussian mean filtering method.

[0080] The above variables are fitted as a quadratic function by using the least square fitting method:

[0081]

[0082]

[0083] Where m is the number of depth values ​​in each column of the mask region, and the objective function is S(i). By taking the partial derivatives of the objective function S with respect to parameters a, b, and c, and then setting the derivatives to zero, the optimal parameter values ​​of the quadratic function f(i) can be obtained.

[0084] The fitting result corresponding to each independent variable is recorded as the standard value, and the depth value d' of each column is used. i,j The samples are fitted to a Gaussian distribution, and the standard deviation of each Gaussian distribution is denoted as σ. i , will d' i,j Normalized to the average value with reference to the standard value, ±σ i Within the upper and lower thresholds, i.e.:

[0085]

[0086] in, For the precise depth value, d' i max and d' i min These are the maximum and minimum depth values ​​obtained by the truncated Gaussian mean filtering method for the adaptive window in column i, respectively.

[0087] After parallax smoothing, a 3D reconstruction is performed on a dense parallax map, such as... Figure 6 As shown, the dense parallax map is converted into a point cloud map, and voxel downsampling is performed on the point cloud map. The voxel size can be selected as 10mm to improve the efficiency of surface reconstruction and limit the number of feature points. The point cloud is rotated so that the target fish body axis is parallel to the x-axis and the contour is perpendicular to the z-axis. This is because the fish poses and angles in the original image are diverse. Rotating it uniformly so that the contour faces the human eye's viewing angle (z-axis) makes it easier to observe intuitively and extract feature points. After the point cloud map is reconstructed through a Poisson surface, a fish body contour map in triangular mesh format is obtained. Surface reconstruction is to obtain the fish's triangular mesh so that feature points can be collected on the triangular mesh.

[0088] like Figure 7 As shown, feature points of the head (P1), tail (P2), back (P3), and abdomen (P4) of the fish body outline are extracted. Feature points are collected on the surface of the triangular mesh, and the distances between all feature points between P1 and P2, and between P3 and P4 are calculated. This yields the fork length and body width of the target fish in the 3D scene. Figures 8-9 As shown; where, fork length is the length of the fish body from the snout to the deepest point of the tail fork, and body width is the distance between the belly and back of the fish body perpendicular to the fork length.

[0089] In this embodiment, the least angle regression method of Lasso regression is used as the multivariate data regression method to fit the body weight, and its cost function is:

[0090]

[0091] wherein w is a weight coefficient, ||·||1 is a 1-norm, and λ is a penalty coefficient.

[0092] A fish body index dataset of known fork length, body width and body weight is constructed in advance, fork length and body width are taken as independent variables, by changing the penalty coefficient, each Cost(w) is compared, and the minimum value is taken, the penalty coefficient of this embodiment is 0.012, the cost function is solved, and the body weight prediction formula is:

[0093] W = 0.00477507fl + 0.0311315wl-4.09581

[0094] wherein fl is fork length, unit: mm, wl is body width, unit: mm, and W is body weight, unit: kg.

[0095] The body weight prediction accuracy is analyzed by root mean square error (RMSE), mean absolute error (MSE) and mean absolute percentage error (MAPE);

[0096]

[0097]

[0098]

[0099] wherein N is the number of images in each group, w i is the true body weight, is the estimated body weight.

[0100] The body weight average prediction value and evaluation index are shown in Table 1.

[0101] Table 1 Body weight average prediction value and evaluation index

[0102]

[0103]

[0104] The embodiment proposes a fish body three-dimensional reconstruction and biomass detection method based on underwater binocular vision, performs instance segmentation on the fish body, realizes three-dimensional reconstruction of the fish body by using a stereo matching algorithm, captures feature points to realize body size calculation, the model production process is also more efficient, and the data efficiency and generalization are also more excellent.

[0105] Embodiment 2

[0106] The embodiment provides an underwater fish body biomass detection system, comprising:

[0107] The instance segmentation module is configured to perform instance segmentation on the underwater binocular image pair to obtain a target fish body binocular image pair, and extract a corresponding mask pair from the target fish body binocular image pair.

[0108] The disparity map processing module is configured to perform stereo matching on the mask pair to obtain a disparity map, and traverse depth values in an adaptive window, and perform truncated Gaussian mean filtering and missing depth value completion on the disparity map until the number of remaining depth values in the window after filtering meets a proportion requirement, thereby obtaining a dense disparity map.

[0109] The biomass detection module is configured to perform three-dimensional reconstruction on the dense disparity map after disparity smoothing processing to obtain a fish body contour, extract fish body feature points in the fish body contour and calculate distances between the fish body feature points, thereby obtaining a fork length and a body width of the target fish body, and estimate the body weight of the target fish body according to a correlation function of the fork length, the body width and the body weight.

[0110] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0111] In more embodiments, there are also provided:

[0112] An electronic device includes a memory and a processor, and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, the method described in Embodiment 1 is completed. For brevity, it will not be repeated here.

[0113] It should be understood that in the embodiments, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSPs, application-specific integrated circuits ASICs, ready-to-program gate arrays FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0114] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, and a part of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0115] A computer readable storage medium for storing computer instructions, when the computer instructions are executed by the processor, the method described in Embodiment 1 is completed.

[0116] The method in the embodiment 1 can be directly embodied by a hardware processor or by a combination of hardware and software modules in the processor. The software modules can be located in a storage medium in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, an electrically erasable programmable memory, a register, or the like. The storage medium is located in a memory, and a processor reads information in the memory and combines hardware to complete the steps of the above method. To avoid repetition, no further detailed description is given herein.

[0117] Those skilled in the art can understand that the units and algorithm steps of the examples described in combination with the embodiments can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software manner depends on the specific application and design constraints of the technical solutions. A person skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0118] Although the specific embodiments of the present application are described above in combination with the drawings, the description is not a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications or changes made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the scope of protection of the present application.

Claims

1. A method for detecting underwater fish biomass, characterized in that, include: After instance segmentation of the underwater binocular image pairs, the target fish binocular image pairs are obtained, and the corresponding mask pairs are extracted from the target fish binocular image pairs. A disparity map is obtained by stereo matching of the mask pairs. The disparity map is then subjected to adaptive window traversal of depth values ​​and truncated Gaussian mean filtering and missing depth value completion until the number of remaining depth values ​​in the window after filtering meets the ratio requirement, thus obtaining a dense disparity map. The truncated Gaussian mean filtering process includes: traversing all depth values ​​with a window of side length N, centered on each pixel in the disparity map. The non-zero depth values ​​within the window are fitted to a Gaussian distribution, and the depth values ​​within the window are filtered using twice the standard deviation of the Gaussian distribution as the truncated mean parameter. After parallax smoothing, the dense parallax map is reconstructed in three dimensions to obtain the fish body outline. The fish body feature points in the fish body outline are extracted and the distance between the fish body feature points is calculated. Thus, the fork length and body width of the target fish body are obtained. Based on the correlation function of fork length, body width and weight, the weight of the target fish body is estimated.

2. The method for detecting underwater fish biomass as described in claim 1, characterized in that, The process of instance segmentation for underwater stereo image pairs includes: Multi-scale attention feature extraction is performed on underwater stereo image pairs. The attention feature maps at each scale are upsampled and then triple attention extraction is performed to generate region candidate boxes for underwater stereo image pairs. The target fish body is detected and segmented based on the region candidate box. The mask is segmented based on the determination of the pixel category within the region candidate box, thereby extracting the target fish body and its corresponding mask.

3. The method for detecting underwater fish biomass as described in claim 1, characterized in that, The process of filling in missing depth values ​​includes: averaging the filtered remaining depth values ​​and replacing the depth values ​​at the filtered points; if the number of filtered remaining depth values ​​does not exceed a set threshold, the window size is expanded until the proportion of remaining depth values ​​meets the requirements, thus obtaining a dense depth map.

4. The method for detecting underwater fish biomass as described in claim 1, characterized in that, The 3D reconstruction process includes: converting the dense parallax map into a point cloud map, voxel downsampling the point cloud map, rotating it so that the target fish body axis is parallel to the X-axis and the contour is perpendicular to the Z-axis, and finally reconstructing it using a Poisson surface to obtain the fish body contour in a triangular mesh format.

5. The method for detecting underwater fish biomass as described in claim 4, characterized in that, Extract the fish body feature points of the head, tail, back, and abdomen outline, and calculate the distances between all feature points of the head and tail, back and abdomen on the triangular mesh surface to obtain the fork length and body width of the target fish body.

6. The method for detecting underwater fish biomass as described in claim 1, characterized in that, By incorporating a cost function for fitting, the correlation functions for fork length, body width, and body weight are obtained as follows: in, fl Fork length, wl For body width, W For body weight.

7. An underwater fish biomass detection system, characterized in that, include: The instance segmentation module is configured to segment underwater binocular image pairs to obtain target fish binocular image pairs, and extract the corresponding mask pairs from the target fish binocular image pairs. The disparity map processing module is configured to perform stereo matching on mask pairs to obtain a disparity map. It then uses an adaptive window to traverse depth values ​​and performs truncated Gaussian mean filtering and missing depth value completion on the disparity map until the number of remaining depth values ​​within the window after filtering meets the proportional requirements, thus obtaining a dense disparity map. The truncated Gaussian mean filtering process includes: traversing all depth values ​​with a window of length N, centered on each pixel in the disparity map; fitting a Gaussian distribution to the non-zero depth values ​​within the window; and filtering the depth values ​​within the window using twice the standard deviation of the Gaussian distribution as the truncated mean parameter. The biomass detection module is configured to perform three-dimensional reconstruction on a dense disparity map after disparity smoothing to obtain the fish body outline, extract fish body feature points from the fish body outline and calculate the distance between fish body feature points, thereby obtaining the fork length and body width of the target fish body, and estimating the weight of the target fish body based on the correlation function of fork length, body width and weight.

8. An electronic device, characterized in that, It includes a memory and a processor, as well as computer instructions stored in the memory and running on the processor, which, when executed by the processor, perform the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Used to store computer instructions, which, when executed by a processor, perform the method described in any one of claims 1-6.

Images