A method for monitoring freshwater shellfish of underwater structures

Abstract

The present invention relates to a method for monitoring freshwater shellfish of underwater structures, comprising: obtaining an image of an underwater structure to be monitored; inputting the image of the underwater structure to be monitored into a freshwater shellfish counting model to obtain a freshwater shellfish density image, wherein the freshwater shellfish counting model is trained by underwater structure image data containing labeled freshwater shellfish, and the freshwater shellfish counting model is constructed using a deep learning network; combining the freshwater shellfish density image with the principle of stereoscopic vision to obtain the number, coverage index, unit weight and biomass index of freshwater shellfish in the image of the underwater structure to be monitored. The data monitored by the present invention is relatively comprehensive, and provides detailed data for a comprehensive understanding of the attachment of freshwater shellfish to underwater structures.

Description

Technical Field

[0001] The invention relates to the technical field of water ecological environment monitoring, and in particular to a method for monitoring freshwater shellfish of underwater structures. Background Art

[0002] The invasion and proliferation of freshwater shellfish will cause serious biofouling problems on the surface of hydraulic structures and inside pipes. The attachment of freshwater shellfish byssus destroys the metal and concrete surface structure of hydraulic components, easily causing pipe blockage and shedding of concrete protective layer, which will harm the strength and durability of concrete structures.

[0003] The prior art proposes a method for calculating the characteristics of freshwater husks based on a deep learning segmentation algorithm. This method can achieve good results in areas where freshwater husks are less distributed. However, in scenes where freshwater husks are densely distributed, the method of estimating the individual size of freshwater husks based on image contours is difficult to achieve ideal detection results. Therefore, the present invention proposes a method for monitoring freshwater husks of underwater structures. Summary of the invention

[0004] The purpose of the present invention is to provide a method for monitoring freshwater shellfish of underwater structures, collect image data of freshwater shellfish on the surface of underwater structures, and then calculate the distribution density map of freshwater shellfish through deep learning counting model, and estimate the number of freshwater shellfish on the image. Finally, the area, coverage and biomass index of the freshwater shellfish distribution area are estimated by using the principle of stereoscopic vision, which provides detailed data for a comprehensive understanding of the attachment of freshwater shellfish on underwater structures.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] A method for monitoring freshwater shellfish of underwater structures, comprising:

[0007] Acquire images of underwater structures to be monitored;

[0008] Inputting the image of the underwater structure to be monitored into a freshwater shellfish counting model to obtain a freshwater shellfish density image, wherein the freshwater shellfish counting model is trained by underwater structure image data containing labeled freshwater shellfish, and the freshwater shellfish counting model is constructed using a deep learning network;

[0009] The density image of freshwater shell vegetables is combined with the principle of stereoscopic vision to obtain the number, coverage index, unit weight and biomass index of freshwater shell vegetables in the image of the underwater structure to be monitored.

[0010] Optionally, obtaining an image of an underwater structure to be monitored includes:

[0011] Collect freshwater shellfish images of underwater structures to obtain binocular images;

[0012] Scaling, Gaussian denoising and first color space conversion of the binocular image to obtain an HSV color image, wherein the first color space conversion is from RGB color space to HSV color space;

[0013] Separate the HSV color image into characteristic channels to obtain H, S, and V channel data;

[0014] Performing contrast stretching operation processing on the V channel data by using contrast-limited adaptive histogram equalization, and performing median filtering processing to obtain processed V channel data;

[0015] The H channel data, the S channel data and the processed V channel data are subjected to channel merging processing and second color space conversion to obtain the image of the underwater structure to be monitored, wherein the second color space conversion is from the HSV color space to the RGB color space.

[0016] Optionally, inputting the image of the underwater structure to be monitored into the freshwater shellfish counting model to obtain the freshwater shellfish density image comprises:

[0017] Cutting the image of the underwater structure to be monitored into a left view and a right view;

[0018] The left view is input into the freshwater shellfish counting model to obtain the freshwater shellfish density image of the left view.

[0019] Optionally, obtaining the number of freshwater shell vegetables in the image of the underwater structure to be monitored includes:

[0020] The density image of the freshwater shell vegetable in the left view is subjected to density map integral calculation to obtain the number of the freshwater shell vegetables.

[0021] Optionally, obtaining the coverage index of freshwater shellfish in the image of the underwater structure to be monitored includes:

[0022] Performing threshold segmentation on the freshwater shellfish density image of the left view to obtain a freshwater shellfish binary image of the left view;

[0023] Performing close morphological processing on the left-view freshwater shellfish binary image to obtain a final left-view freshwater shellfish binary image;

[0024] Obtaining the number of white pixels in the final left-view freshwater shellfish binary image, wherein the number of white pixels in the final left-view freshwater shellfish binary image is the freshwater shellfish pixel area in the left-view;

[0025] The coverage index of the freshwater shellfish is obtained according to the pixel area of ​​the freshwater shellfish in the left view and the total pixel area of ​​the freshwater shellfish binary image in the left view.

[0026] Optionally, obtaining the unit weight of freshwater shellfish in the image of the underwater structure to be monitored includes:

[0027] Perform binocular matching on the left view and the right view to obtain a disparity image of the left view;

[0028] Filtering the disparity image to obtain a dense disparity image;

[0029] Based on the dense disparity image, construct three-dimensional point cloud data;

[0030] Matching the three-dimensional point cloud data with white pixel points in the final left-view freshwater shell vegetable binary image to obtain the coverage area of ​​the freshwater shell vegetable;

[0031] The unit weight of the freshwater shell vegetables is obtained according to the coverage area and number of the freshwater shell vegetables.

[0032] Optionally, obtain freshwater shellfish coverage including:

[0033] Matching the three-dimensional point cloud data with white pixel points in the final left-view freshwater shellfish binary image, obtaining the maximum enclosing rectangle of the white area in the final left-view freshwater shellfish binary image and the proportion of white pixels in the maximum enclosing rectangle;

[0034] The coverage area of ​​the freshwater shell vegetable is obtained according to the maximum enclosing rectangle and the proportion of white pixels in the maximum enclosing rectangle.

[0035] Optionally, obtaining the unit weight of the freshwater shell vegetables according to the coverage area and number of the freshwater shell vegetables comprises:

[0036] Divide the coverage area of ​​the freshwater shell vegetables by the number of the freshwater shell vegetables to obtain the average area of ​​the freshwater shell vegetables;

[0037] If the average area is greater than a first threshold, multiplying the first preset weight by a first size weight coefficient to obtain a unit weight of the freshwater shell vegetable, wherein the first size weight coefficient is obtained by a ratio of the average area to the first threshold;

[0038] If the average area is less than the first threshold and greater than the second threshold, multiplying the second preset weight by a second size weight coefficient to obtain a unit weight of the freshwater shell vegetable, wherein the second size weight coefficient is obtained by the ratio of the average area to the second threshold;

[0039] If the average area is smaller than a third threshold, the third preset weight is multiplied by a third size weight coefficient to obtain the unit weight of the freshwater shell vegetable, wherein the third size weight coefficient is obtained by the ratio of the average area to the third threshold.

[0040] Optionally, obtaining the biomass index of freshwater shellfish in the image of the underwater structure to be monitored includes:

[0041] Obtaining a total weight according to the number and unit weight of the freshwater shell vegetables;

[0042] Matching the three-dimensional point cloud data with white pixel points in the final left-view freshwater shellfish binary image to obtain an average distance between the camera and the freshwater shellfish;

[0043] According to the measured distance between the camera and the freshwater shellfish and the corresponding area, a distance-area relationship model is established;

[0044] Acquire the actual monitoring area according to the average distance and the relationship model between the distance and the area;

[0045] The biomass index of the freshwater shell vegetable is obtained according to the total weight and the actual monitoring area.

[0046] The beneficial effects of the present invention are:

[0047] The present invention proposes a method for monitoring freshwater shellfish on underwater structures, which can automatically and intelligently calculate monitoring indicators of freshwater shellfish on underwater structures, and provide technical support for the normalized monitoring, early warning and prevention of freshwater shellfish.

[0048] A freshwater shellfish monitoring method for underwater structures developed by the present invention utilizes deep learning technology and stereoscopic vision principles to not only monitor the coverage index of freshwater shellfish, but also obtain indicators such as the distribution, number, average individual size, area, unit weight and biomass of freshwater shellfish. The monitored data is relatively comprehensive, providing detailed data for a comprehensive understanding of the attachment of freshwater shellfish to underwater structures.

[0049] The statistics of the number of freshwater shell vegetables in the present invention adopts a deep learning counting model, which can effectively solve the deviations caused by the dense distribution in the picture and the statistics of the number of freshwater shell vegetables of different sizes, and improve the accuracy of indicators such as average individual size, area, and unit weight. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A flow chart of a method for monitoring freshwater shellfish of an underwater structure according to an embodiment of the present invention;

[0052] Figure 2 This is an offline modeling flow chart of an embodiment of the present invention;

[0053] Figure 3 is an image enhancement flow chart of an embodiment of the present invention;

[0054] Figure 4 A flow chart of counting the number of freshwater shellfish according to an embodiment of the present invention;

[0055] Figure 5 A flow chart of calculating the coverage index of freshwater shellfish according to an embodiment of the present invention;

[0056] Figure 6 A flow chart of estimating the weight of a freshwater shellfish dish according to an embodiment of the present invention;

[0057] Figure 7 A flow chart of calculating the area of ​​freshwater shellfish according to an embodiment of the present invention;

[0058] Figure 8 A flow chart of calculating the unit weight of freshwater shellfish according to an embodiment of the present invention;

[0059] Fig. 9 It is a flow chart of calculating the biomass index of freshwater shellfish according to an embodiment of the present invention;

[0060] Fig.10 The flowchart of calculating the camera monitoring area according to an embodiment of the present invention is shown in FIG. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0062] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0063] Embodiment 1

[0064] This embodiment provides a method for monitoring freshwater shellfish of an underwater structure, comprising:

[0065] Acquire images of underwater structures to be monitored;

[0066] Inputting the underwater structure image to be monitored into a freshwater shellfish counting model to obtain a freshwater shellfish density image, wherein the freshwater shellfish counting model is trained by underwater structure image data containing labeled freshwater shellfish;

[0067] Specifically, they include:

[0068] (1) Training a freshwater shellfish counting model based on deep learning:

[0069] Collect and annotate underwater structure image data containing freshwater shellfish to form original training sample data;

[0070] Performing automatic data augmentation processing on the original training sample data to form an augmented data set;

[0071] The original training samples are merged with the augmented data set to form the freshwater shellfish counting model training samples;

[0072] Select and train the deep learning counting model;

[0073] Test the counting effect of freshwater shellfish counting model;

[0074] Adjust the training samples, annotation methods, or add new training samples based on the test results of the trained model;

[0075] Iterate and optimize the training model until the accuracy meets the expected target.

[0076] (2) Perform image enhancement processing on the collected underwater structure images:

[0077] Scaling the acquired binocular images to improve the algorithm's computational efficiency;

[0078] Perform Gaussian denoising and color space conversion on the scaled binocular image;

[0079] The contrast-limited adaptive histogram equalization (CLAHE) algorithm is used to perform contrast stretching on the processed binocular image.

[0080] Median filtering is performed on the contrast-enhanced binocular image to eliminate the noise generated during the contrast stretching process.

[0081] (3) Perform freshwater shellfish counting model inference on the enhanced underwater structure image to obtain the freshwater shellfish density image:

[0082] Split a single binocular image into two images: left view and right view;

[0083] The freshwater shellfish counting model is inferred on the left view image to obtain the freshwater shellfish density map based on the left view.

[0084] The density image of freshwater shell vegetables is combined with the principle of stereoscopic vision to obtain the number, coverage index and unit weight of freshwater shell vegetables in the image of the underwater structure to be monitored.

[0085] Specifically, based on the freshwater shell vegetable density image, counting the number includes: performing density map integral calculation on the freshwater shell vegetable density map based on the left view to obtain the number of freshwater shell vegetables in the image.

[0086] Binarization segmentation and morphological processing are performed on the density image of freshwater shellfish to obtain the segmented image of freshwater shellfish, and its coverage index is calculated including:

[0087] Perform threshold segmentation on the freshwater shellfish density image of the left view to obtain a freshwater shellfish binary image of the left view;

[0088] Apply morphological close processing on the left view freshwater shellfish binary image to connect the disconnected boundaries to form the final left view freshwater shellfish binary image;

[0089] The pixel area of ​​the left view is known. The number of white pixels is counted on the final left view freshwater shellfish binary image. This number is the pixel area of ​​the freshwater shellfish in the left view. The coverage index of the freshwater shellfish can be obtained by dividing this area by the pixel area of ​​the left view.

[0090] Using the freshwater shellfish segmentation image and stereo vision technology, the unit weight of the freshwater shellfish in the current image is estimated to include:

[0091] On the collected binocular image data, binocular matching is performed with the left view as the reference to obtain the disparity image of the left view;

[0092] Reconstructing the 3D point cloud data of the left view on the disparity image of the left view;

[0093] The three-dimensional data (X and Y coordinates) corresponding to the white pixel points of the binary image of the freshwater shellfish in the final left view are counted on the three-dimensional point cloud data of the left view, and the maximum enclosing rectangular area of ​​the measurement is calculated;

[0094] Calculate the maximum bounding rectangle of the white area on the final left-view freshwater shellfish binary image;

[0095] Count the proportion of white pixels in the largest enclosing rectangle in the final left-view freshwater shellfish binary image (the proportion of freshwater shellfish);

[0096] The area of ​​the measured maximum enclosing rectangle is multiplied by the proportion of white pixels in the maximum enclosing rectangle (the proportion of freshwater shellfish) to estimate the area of ​​the freshwater shellfish in the actual scene;

[0097] The estimated area of ​​freshwater shellfish in the actual scenario is divided by the calculated number of freshwater shellfish to obtain the average area of ​​freshwater shellfish;

[0098] The unit weight of the freshwater shellfish was calculated based on the average area of ​​the freshwater shellfish.

[0099] Combining the unit weight and number of freshwater shell vegetables, the biomass index is calculated, including:

[0100] According to the unit weight of the freshwater shell vegetables and the number of the freshwater shell vegetables, the total weight of the freshwater shell vegetables in the left view is obtained;

[0101] The three-dimensional data (Z coordinate) corresponding to the white pixel points of the binary image of the freshwater shellfish in the final left view is counted on the three-dimensional point cloud data of the left view, and the average distance between the camera and the freshwater shellfish is calculated;

[0102] According to the area calibrated offline at different distances, combined with the calculated average distance between the camera and the freshwater shellfish, the actual monitoring area of ​​the current camera (left view) is estimated;

[0103] The calculated total weight of the freshwater shellfish in the left view is divided by the actual monitoring area of ​​the left view to obtain the biomass index of the freshwater shellfish.

[0104] Embodiment 2

[0105] like Figure 1 As shown, the present invention discloses a method for monitoring freshwater shellfish of underwater structures, comprising the following steps:

[0106] (1) Offline modeling: Training a dense freshwater shellfish counting model based on deep learning.

[0107] (2) Image enhancement: De-noising and contrast enhancement of the collected underwater binocular image data.

[0108] (3) Counting the number of freshwater shell vegetables: Perform freshwater shell vegetable counting model inference on the enhanced image to calculate the number of freshwater shell vegetables.

[0109] (4) Calculation of freshwater shellfish coverage index: Calculate the coverage area of ​​freshwater shellfish and its coverage index based on the freshwater shellfish density map inferred from the freshwater shellfish counting model.

[0110] (5) Estimating the weight of a piece of freshwater shellfish: Using the principle of stereoscopic vision, the area of ​​the freshwater shellfish is estimated, and then combined with the number of freshwater shellfish, its average size and unit weight are calculated.

[0111] (6) Calculate the biomass index of freshwater shellfish: First, calculate the total weight of freshwater shellfish in the monitoring area by the weight of each unit of freshwater shellfish, then combine the distance information to obtain the area of ​​the monitoring area, and finally calculate the biomass index of freshwater shellfish.

[0112] like Figure 2 As shown, the offline modeling process includes:

[0113] (1) Collect and annotate image data of underwater structures containing freshwater shellfish to form original training sample data.

[0114] (2) Perform automated data augmentation processing on the original training sample data to form an augmented data set.

[0115] (3) The original training samples are combined with the augmented dataset to form the training samples for the freshwater shellfish counting model.

[0116] (4) A deep learning counting model is selected, and the p2pnet model is preferably used for freshwater shellfish counting model training.

[0117] (5) Test the counting effect of the freshwater shellfish counting model;

[0118] (6) Adjust the training samples, labeling methods, or add new training samples based on the test results of the trained model.

[0119] (7) Iteratively optimize the training model until the accuracy meets the expected target.

[0120] like Figure 3 As shown, the image enhancement process includes:

[0121] (1) In order to improve the operating efficiency of the algorithm, the acquired binocular image is first scaled to obtain ResizedImg, and the preferred scaling size is 1920*540 resolution.

[0122] (2) Perform Gaussian denoising on the ResizedImg image (in-place operation).

[0123] (3) Perform color space conversion on ResizedImg after Gaussian denoising to facilitate subsequent calculation and processing. Preferably, ResizedImg is converted from RGB color space to HSV color space to obtain ResizedHSVImg. HSV feature extraction is a commonly used image processing technology, which converts color images from RGB color space to HSV color space and then extracts the feature values ​​of the three channels H, S, and V. In HSV space, the H channel represents the hue of the color, the S channel represents the saturation of the color, and the V channel represents the brightness of the color.

[0124] (4) Separate the three channels into HImg, SImg and VImg.

[0125] (5) Apply the CLAHE (contrast limited adaptive histogram equalization) algorithm to perform contrast stretching operation on VImg to obtain VImg1.

[0126] (6) Perform median filtering (in-place operation) on VImg1.

[0127] (7) HImg, SImg and VImg1 are merged into a three-channel ResizedHSVImg1.

[0128] (8) Convert ResizedHSVImg1 from the HSV color space to the RGB color space to obtain ResizedImg1.

[0129] like Figure 4 As shown, the process of counting the number of freshwater shell vegetables includes:

[0130] (1) Since ResizedImg1 is spliced ​​left and right, ResizedImg1 is divided into the left view LeftImg and the right view RightImg.

[0131] (2) Input LeftImg into the freshwater shellfish counting model to calculate the density map DensityImg of the freshwater shellfish in the image.

[0132] (3) Perform density map integral calculation on DensityImg to obtain the number num of freshwater shell vegetables in the image.

[0133] like Figure 5 As shown, the process of calculating the coverage index of freshwater shellfish includes:

[0134] (1) Perform threshold segmentation on DensityImg to obtain the left view freshwater shellfish binary image BinaryImg.

[0135] (2) Apply morphological close processing (in-place operation) to BinaryImg to connect the disconnected boundaries.

[0136] (3) Count the number of white pixels in BinaryImg, which is the pixel area areaT of the freshwater shellfish in the left view.

[0137] (4) By dividing areaT by the total pixel area of ​​BinaryImg, the coverage index of freshwater shell vegetables relative to the entire left view monitoring area can be obtained.

[0138] like Figure 6 As shown, the process of estimating the unit weight of freshwater shell includes:

[0139] (1) Performing binocular image matching operation on the left view LeftImg and the right view RightImg. Preferably, using SGBM (semi-global block matching) algorithm to calculate the disparity image disparityImg.

[0140] (2) Perform WLS (weighted least squares) filtering on disparityImg to obtain a dense disparity image disparityWLSImg.

[0141] (3) Based on disparityWLSImg and the calibration parameters of the left camera of the binocular camera, the dense three-dimensional point cloud data LeftImg3D is calculated.

[0142] (4) Traverse and analyze the LeftImg3D data corresponding to the white pixels in BinaryImg to calculate the coverage area of ​​freshwater shell vegetables.

[0143] (5) The unit weight of the freshwater shellfish is calculated based on the coverage area of ​​the freshwater shellfish and the number of the freshwater shellfish.

[0144] like Figure 7 As shown, the process of calculating the area of ​​freshwater shellfish includes:

[0145] (1) Traverse and analyze the white pixels (the area of ​​freshwater shellfish) in BinaryImg, find the minimum X and Y pixel coordinates (x1, x2) and the maximum X and Y pixel coordinates (x2, y2), and construct the maximum enclosing rectangle Rect of the freshwater shellfish on BinaryImg. Rect is a structure consisting of four variables: x, y, width, and height. x represents the minimum X pixel coordinate x1; y represents the minimum Y pixel coordinate y1; width represents the width of the rectangle, and the calculation formula is x2-x1; height represents the height of the rectangle, and the calculation formula is y2-y1.

[0146] (2) Count the number of white pixels in BinaryImg, then calculate the pixel area (width*height) of the maximum enclosing rectangular box Rect, and finally divide Count by areaR to obtain the coverage rate of freshwater shell vegetables relative to the maximum enclosing rectangular box Rect.

[0147] (3) Traverse and analyze the X-coordinate and Y-coordinate data of LeftImg3D corresponding to the white pixel in BinaryImg, and find the rectangular box Rect3D on LeftImg3D corresponding to the largest enclosing rectangular box Rect. Rect3D and Rect have the same structure. By calculating the product of width and height in Rect3D, the measured area areaR3D of Rect3D can be obtained.

[0148] (4) By multiplying area3D by the coverage rate of freshwater shellfish, the area of ​​freshwater shellfish in the actual scene can be estimated.

[0149] like Figure 8 As shown, the process of calculating the unit weight of freshwater shellfish includes the following contents:

[0150] (1) Divide the estimated area of ​​freshwater shell vegetables in the actual scenario by the calculated number of freshwater shell vegetables num to obtain the average area of ​​freshwater shell vegetables areaMean.

[0151] (2) According to the actual scene investigation, a plurality of different freshwater shell vegetable size and weight data are manually given, and the given freshwater shell vegetable size and weight data are manually measured and weighed. Preferably, in this scheme, only three types of freshwater shell vegetable size and weight data, large, medium and small, are given. The area of ​​large freshwater shell vegetable is recorded as bm, and the weight is recorded as bg; the area of ​​medium freshwater shell vegetable is recorded as mm, and the weight is recorded as mg; the area of ​​small freshwater shell vegetable is recorded as sm, and the weight is recorded as sg.

[0152] (3) Determine whether the estimated average area of ​​freshwater shellfish areaMean is greater than bm. If it is greater than bm, calculate the size weight coefficient R, which is calculated by the formula R = areaMean / bm; then multiply bg by R to obtain the unit weight weightM of the freshwater shellfish on the current screen. If areaMean is less than bm, further determine the size of areaMean and mm.

[0153] (4) Determine whether the estimated average area of ​​freshwater shell vegetables areaMean is greater than mm. If it is greater than mm, calculate the size weight coefficient R, and its calculation formula is R=areaMean / mm; then multiply mg by R to obtain the unit weight weightM of the freshwater shell vegetables on the current screen.

[0154] (5) If the average area of ​​the freshwater shellfish areaMean is less than mm, calculate the size weight coefficient R relative to sm, and its calculation formula is R=areaMean / sm; then multiply sg by R to obtain the unit weight weightM of the freshwater shellfish on the current screen.

[0155] like Fig. 9 As shown, the process of calculating the biomass index of freshwater shellfish includes the following contents:

[0156] (1) Multiply the unit weight weightM of the freshwater shellfish by the number num of freshwater shellfish to calculate the total weight weightT of the freshwater shellfish in the monitoring area.

[0157] (2) Traverse and analyze the LeftImg3D data corresponding to the white pixels in BinaryImg, and calculate the average distance Dist between the freshwater shellfish and the camera.

[0158] (3) According to the calculation formula between the camera monitoring area and distance calibrated in the offline scene, the camera monitoring area areaC at the current distance Dist between the freshwater shellfish and the camera is calculated.

[0159] (4) Divide the total weight weightT of the freshwater shellfish in the monitoring area by the monitoring area areaC of the camera to obtain the biomass index of the freshwater shellfish in the current monitoring area.

[0160] like Fig.10 As shown, the process of calculating the camera monitoring area includes the following contents:

[0161] (1) In order to collect clear image data of freshwater shellfish under the condition of supplementary lighting, the maximum distance between the camera and the freshwater shellfish is generally no more than 1 meter, and the minimum distance is no less than 20 cm.

[0162] (2) In the terrestrial environment, 9 sampling points were constructed starting from 20 cm and at intervals of 10 cm until 100 cm.

[0163] (3) The position of each sampling point is located by a laser ranging device. The camera is pointed at a fixed wall and the actual width and height data of the camera in the left view are recorded by manually pulling a tape measure, and its monitoring area is calculated.

[0164] (4) A data set consisting of 9 sets of two-dimensional data is constructed, where the horizontal axis represents the distance between the camera and the target, and the vertical axis represents the monitoring area of ​​the point. The least squares curve fitting is then performed to obtain the fitting function f(x).

[0165] (5) Substitute the calculated average distance Dist between the freshwater shellfish and the camera into f(x) to obtain the camera monitoring area areaC at the current distance.

[0166] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A method for monitoring freshwater shellfish in underwater structures, characterized in that: include: Acquire images of underwater structures to be monitored; Inputting the image of the underwater structure to be monitored into a freshwater shellfish counting model to obtain a freshwater shellfish density image, wherein the freshwater shellfish counting model is trained by underwater structure image data containing labeled freshwater shellfish, and the freshwater shellfish counting model is constructed using a deep learning network; Inputting the image of the underwater structure to be monitored into the freshwater shellfish counting model to obtain the freshwater shellfish density image comprises: Cutting the image of the underwater structure to be monitored into a left view and a right view; Inputting the left view into the freshwater shellfish counting model to obtain a freshwater shellfish density image of the left view; The density image of freshwater shell vegetables is combined with the principle of stereoscopic vision to obtain the number, coverage index, unit weight and biomass index of freshwater shell vegetables in the image of the underwater structure to be monitored; Obtaining the coverage index of freshwater shellfish in the image of the underwater structure to be monitored includes: Performing threshold segmentation on the freshwater shellfish density image of the left view to obtain a freshwater shellfish binary image of the left view; Performing close morphological processing on the left-view freshwater shellfish binary image to obtain a final left-view freshwater shellfish binary image; Obtaining the number of white pixels in the final left-view freshwater shellfish binary image, wherein the number of white pixels in the final left-view freshwater shellfish binary image is the freshwater shellfish pixel area in the left-view; Obtaining a coverage index of the freshwater shellfish according to the pixel area of ​​the freshwater shellfish in the left view and the total pixel area of ​​the freshwater shellfish binary image in the left view; Obtaining the unit weight of freshwater shellfish in the image of the underwater structure to be monitored includes: Perform binocular matching on the left view and the right view to obtain a disparity image of the left view; Filtering the disparity image to obtain a dense disparity image; Based on the dense disparity image, construct three-dimensional point cloud data; Matching the three-dimensional point cloud data with white pixel points in the final left-view freshwater shell vegetable binary image to obtain the coverage area of ​​the freshwater shell vegetable; The unit weight of the freshwater shell vegetables is obtained according to the coverage area and number of the freshwater shell vegetables.

2. The method for monitoring freshwater shellfish of underwater structures according to claim 1, characterized in that: Obtaining images of underwater structures to be monitored includes: Collect images of freshwater shellfish of underwater structures to obtain binocular images; Scaling, Gaussian denoising and first color space conversion of the binocular image to obtain an HSV color image, wherein the first color space conversion is from RGB color space to HSV color space; Separate the HSV color image into characteristic channels to obtain H, S, and V channel data; Performing contrast stretching operation processing on the V channel data by using a contrast-limited adaptive histogram equalization algorithm, and performing median filtering processing to obtain processed V channel data; The H channel data, the S channel data and the processed V channel data are subjected to channel merging processing and second color space conversion to obtain the image of the underwater structure to be monitored, wherein the second color space conversion is from the HSV color space to the RGB color space.

3. The method for monitoring freshwater shellfish of underwater structures according to claim 1, characterized in that: Obtaining the number of freshwater shell vegetables in the image of the underwater structure to be monitored includes: The density image of the freshwater shell vegetable in the left view is subjected to density map integral calculation to obtain the number of the freshwater shell vegetables.

4. The method for monitoring freshwater shellfish of underwater structures according to claim 1, characterized in that: Get freshwater shellfish coverage including: Matching the three-dimensional point cloud data with white pixel points in the final left-view freshwater shellfish binary image, obtaining the maximum enclosing rectangle of the white area in the final left-view freshwater shellfish binary image and the proportion of white pixels in the maximum enclosing rectangle; The coverage area of ​​the freshwater shell vegetable is obtained according to the maximum enclosing rectangle and the proportion of white pixels in the maximum enclosing rectangle.

5. The method for monitoring freshwater shellfish of underwater structures according to claim 4, characterized in that: According to the coverage area and number of the freshwater shell vegetables, obtaining the unit weight of the freshwater shell vegetables comprises: Divide the coverage area of ​​the freshwater shell vegetables by the number of the freshwater shell vegetables to obtain the average area of ​​the freshwater shell vegetables; If the average area is greater than a first threshold, multiplying the first preset weight by a first size weight coefficient to obtain a unit weight of the freshwater shell vegetable, wherein the first size weight coefficient is obtained by a ratio of the average area to the first threshold; If the average area is less than the first threshold and greater than the second threshold, multiplying the second preset weight by a second size weight coefficient to obtain a unit weight of the freshwater shell vegetable, wherein the second size weight coefficient is obtained by the ratio of the average area to the second threshold; If the average area is smaller than a third threshold, the third preset weight is multiplied by a third size weight coefficient to obtain the unit weight of the freshwater shell vegetable, wherein the third size weight coefficient is obtained by the ratio of the average area to the third threshold.

6. The method for monitoring freshwater shellfish of underwater structures according to claim 4, characterized in that: Obtaining the biomass index of freshwater shellfish in the image of the underwater structure to be monitored includes: Obtaining a total weight according to the number and unit weight of the freshwater shell vegetables; Matching the three-dimensional point cloud data with white pixel points in the final left-view freshwater shellfish binary image to obtain an average distance between the camera and the freshwater shellfish; According to the measured distance between the camera and the freshwater shellfish and the corresponding area, a distance-area relationship model is established; Acquire the actual monitoring area according to the average distance and the relationship model between the distance and the area; The biomass index of the freshwater shell vegetable is obtained according to the total weight and the actual monitoring area.

Images