A monitoring system and a monitoring method for three-dimensional reconstruction of underwater fish school

Abstract

This application discloses a monitoring system and method for 3D reconstruction of underwater fish schools. The monitoring system includes a data acquisition module, a data processing module, a 3D point cloud fusion module, and a 3D reconstruction module. The data processing module parses raw sonar data into image data and depth sounding data, and performs data processing. The 3D point cloud fusion module constructs a 3D point cloud, performs point cloud registration and feature fusion, and constructs a local 3D feature image of the fish school. The 3D reconstruction module fuses the formed local 3D feature images of the fish school to present the overall 3D outline of individual fish and the distribution status of the fish school in 3D space. This application combines point cloud matching and feature matching to achieve 3D reconstruction of underwater fish school distribution, constructing a global, continuous, accurate, and robust real-time 3D dynamic fish school distribution map of a large-scale scene, reducing image distortion caused by insufficient detection.

Description

Technical Field

[0001] This application relates to the field of data processing, specifically to a monitoring system and method for three-dimensional reconstruction of underwater fish schools. Background Technology

[0002] The current marine aquaculture industry mainly adopts extensive aquaculture methods based on net cages. This makes it difficult for aquaculture workers to obtain information about the activity of fish schools underwater, resulting in low aquaculture efficiency and poor quality. Currently, underwater fish school detection is mainly done using sonar. However, this method relies entirely on sonar and mathematical processing. Underwater fish schools are complex, and the echoes from a single sonar receiver are very chaotic, easily affected by the external environment, and cannot effectively distinguish between fish and other objects, let alone identify the size and species of fish. Furthermore, most methods cannot reconstruct the complete fish school distribution. In addition, existing technologies also rely on cameras and underwater cameras to monitor the distribution of underwater fish schools, but they cannot accurately reconstruct the three-dimensional distribution of underwater fish schools. Moreover, the equipment is complex and limited by shooting distance and lighting conditions. Summary of the Invention

[0003] The purpose of this application is to provide a monitoring system and method for three-dimensional reconstruction of underwater fish schools. This application combines sonar two-dimensional images with depth sounding data, and uses point cloud matching and feature matching methods to reconstruct the three-dimensional contour features and distribution of underwater fish schools, thus solving the aforementioned technical problems.

[0004] This application provides a monitoring system for three-dimensional reconstruction of underwater fish schools, including a data acquisition module, a data processing module, a three-dimensional point cloud fusion module, and a three-dimensional reconstruction module;

[0005] The data acquisition module includes a sonar transmitting unit and a sonar receiving unit. The sonar receiving unit is used to acquire the raw sonar data of the sampling point, and the raw sonar data is transmitted to the data processing module.

[0006] The data processing module is used to parse the original sonar data of the sampling points into image data and depth measurement data, and to perform data processing: combining the number of Pings M and the number of sampling points N in each Ping into an M×N image, converting each Ping sonar data into the corresponding grayscale value and storing it in the image to obtain the original sonar image.

[0007] The 3D point cloud fusion module is used to construct a 3D point cloud and perform point cloud registration and feature fusion to form a local 3D feature image of the fish school.

[0008] The three-dimensional reconstruction module is used to fuse the local three-dimensional feature images of the fish school to present the three-dimensional outline of the individual fish and the distribution of the fish school in three-dimensional space.

[0009] In some embodiments, the original sonar image is sampled using a local median sampling method to reduce the redundancy of the image data.

[0010] In some embodiments, the local median sampling method includes the following steps: scanning the extracted image data through a 5×t local window and calculating the mean of the corresponding grid as the final image pixel value to obtain the sampled sonar image, where t is the sampling rate of the image in the column direction.

[0011] In some embodiments, the three-dimensional point cloud data is established using the following method:

[0012] The X-axis coordinate of the point cloud data is: X = sin(pitch) × L × sinα

[0013] The Y-axis coordinate of the point cloud data is: Y = sin(pitch) × L × cosα

[0014] The Z-axis coordinate of the point cloud data is: Z = -range × sinθ × cos(pitch) - depth

[0015] Where pitch is the angle between the sonar transmitting unit and the XOZ plane, α is the angle between the sonar transmitting unit and the YOZ plane, range is the slant distance corresponding to each sampling point in the depth measurement data in each Ping data, θ is the angle between each sampling point in the depth measurement data in each Ping data and the XOY plane, and depth is the depth of the corresponding sonar transmitting unit from the water surface.

[0016] In some embodiments, color information is assigned to the three-dimensional point cloud: a two-dimensional coordinate system is established with the horizontal distance of the sonar image sampling points as the abscissa and the gray value as the ordinate. Based on the two-dimensional coordinate system, the depth information corresponding to some sampling points is found by interpolation of the depth measurement data, and the corresponding color information is assigned to each depth measurement sampling point.

[0017] In some embodiments, the three-dimensional point cloud is processed as follows: the depth measurement data and the image data are placed in the same coordinate system according to the horizontal distance and cubic spline interpolation is performed.

[0018] In some embodiments, information from the image data is interpolated to obtain information from the corresponding depth measurement data, and the grayscale value and horizontal distance of the image are interpolated based on the information from the depth measurement data.

[0019] In some embodiments, the cubic spline interpolation method includes the following steps:

[0020] The grayscale values ​​corresponding to the image data of the sonar sampling points are used as nodes. Each node consists of (x, y), and there are a total of n+1 nodes, represented as (x0, y0)...(x... ... n y n ),

[0021] S i (x)=a i +b i (xx i )+c i (xx i ) 2 +d i (xx i ) 3

[0022] Where x represents the horizontal distance pos corresponding to the sampling point, y represents the gray value corresponding to the sampling point, and S i (x) represents the spline curve defined at the i-th node, where i = 0, 1, ..., n-1;

[0023] Get a for each piecewise function i b i c i d i Ultimately, S was obtained. i The (x) function is used to interpolate the sampling points of the depth sounding data.

[0024] In some embodiments, the 3D point cloud fusion module includes the fusion of two point clouds to construct a 3D surface of a fish school.

[0025] In some embodiments, the two point clouds are fused by the following method: the image is flipped and translated by the point cloud registration method of ICP so that the two point clouds are aligned in the same coordinate system, and the average value of the overlapping data of the two point clouds is taken.

[0026] In some embodiments, after averaging the overlapping data of two point clouds, the color parameters of the image are added to the three-dimensional spatial distance to transform the three-dimensional space into a four-dimensional space. The spatial information is the main focus, and the color information is the secondary focus. Different weights are assigned between [0,1] to achieve precise alignment between point clouds.

[0027] In some embodiments, the 3D point cloud fusion module includes the construction of a 3D point cloud, point cloud registration, and feature fusion. The horizontal distance between the image data sampling points and the depth sounding data sampling points is calculated based on the sampling frequency of the sonar transmitting unit and the time interval between the sonar receiving unit receiving echo signals. A two-dimensional coordinate system is established with the horizontal distance of the sonar image sampling points as the abscissa and the grayscale value as the ordinate. Based on the two-dimensional coordinate system, depth information corresponding to some sampling points is found by interpolation of the depth sounding data, and corresponding color information is assigned to each depth sounding sampling point to construct a 3D point cloud of the depth sounding data. The 3D point cloud data is then fused using point cloud registration and feature fusion techniques to form a local 3D feature image of the fish school.

[0028] In some embodiments, the three-dimensional reconstruction module is used to fuse the local three-dimensional feature images of the fish school constructed by the data acquisition module respectively, so as to fully present the body shape features and underwater distribution of the fish school in three-dimensional space.

[0029] In some embodiments, the 3D reconstruction module includes the integration and stitching of multiple local contour images of sonar fish swarms.

[0030] Accordingly, this application provides a method for three-dimensional reconstruction of an underwater fish school, including the following steps:

[0031] (S1) The sonar transmitting unit in the data acquisition module transmits a detection signal, the sonar receiving unit receives the echo signal, and then transmits the signal to the data processing module.

[0032] (S2) The data processing module extracts image data and depth data from the echo signal and performs data processing: the number of Pings M and the number of sampling points N in each Ping are combined into an M×N image, each Ping sonar data is converted into the corresponding gray value and stored in the image to obtain the original sonar image, and the original sonar image is sampled using the local median sampling method to reduce the redundancy of the data.

[0033] (S3) The sonar image data obtained in step (S2) is converted to a two-dimensional coordinate system with the sampling point as the abscissa and the gray value as the ordinate. Three-dimensional point cloud data is established through the three-dimensional point cloud fusion module. Color information is assigned to the three-dimensional point cloud data according to the established two-dimensional coordinate system. Point cloud registration and feature fusion are performed on the three-dimensional point cloud data to obtain the local three-dimensional contour surface of the fish school.

[0034] (S4) The three-dimensional local surface of the fish school established in step (S3) is stitched together to obtain the complete three-dimensional outline of the individual fish and the underwater distribution map of the fish school.

[0035] The beneficial effects of this application are as follows: This application provides a monitoring system for the three-dimensional reconstruction of underwater fish schools, including a data acquisition module, a data processing module, a three-dimensional point cloud fusion module, and a three-dimensional reconstruction module. This application reconstructs the three-dimensional distribution of fish schools through the three-dimensional point cloud module and the three-dimensional reconstruction module, constructing a global, continuous, consistent, accurate, and robust large-scale three-dimensional fish school map, reducing image distortion caused by insufficient detection. This application constructs a two-dimensional sonar image from the original extracted data. Its color information reflects the echo intensity of the underwater fish school, corresponding to the contents of the net cage. The three-dimensional point cloud reflects the distribution of the underwater fish school, representing the distribution changes of the fish school on the seabed. By fusing the two-dimensional sonar image with the three-dimensional point cloud, and assigning the constructed three-dimensional point cloud the color information corresponding to the sonar image, a colored three-dimensional point cloud is constructed. The colored three-dimensional point cloud image will contain richer underwater fish school information, capable of simultaneously representing distribution and fish school contour features. Attached Figure Description

[0036] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 Flowchart of a 3D reconstruction system for a school of fish. Detailed Implementation

[0038] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. In addition, in the description of this application, the term "comprising" means "including but not limited to". The terms first, second, third, etc. are used only as illustrative purposes and do not impose numerical requirements or establish an order.

[0039] Traditional methods for fusing sonar images and depth sounding data rely on registering and fusing features from both sonar and 2D depth images. These methods require high-quality features from both, making them difficult to implement when features are limited. This application provides a system for 3D reconstruction of underwater fish schools, comprising a data acquisition module, a data processing module, a 3D point cloud fusion module, and a 3D reconstruction module. This application converts depth images into 3D point cloud images and achieves initial data fusion through the transformation relationship between the spatial positions of the two point cloud data sets. Further fusion is then performed considering image features, avoiding the problem of poor fusion results due to limited features. The 3D image constructed in this application is smoother.

[0040] In this embodiment, the data acquisition module includes a sonar transmitting unit and a sonar receiving unit. The sonar receiving unit is used to acquire raw sonar data, and the raw sonar data is transmitted to the data processing module.

[0041] In this embodiment, the arrangement of the sonar probes in the sonar transmitting unit is not limited, as long as the sonar can completely cover the activity range of the fish. As an optional implementation, the sonar transmitting unit consists of four sonar probes with a transmission frequency of 200Hz and a pulse angle between 20° and 30°. Considering the distribution range of the fish in the water, two sonar probes are arranged diagonally on the upper side of the net cage, pointing downwards at a 28° angle to the horizontal. Two more sonar probes are symmetrically placed at a 10° angle to the water surface, one-fifth of the way down from the bottom of the net cage, so that the sonar can completely cover the activity range of the fish. At the same time, the four sonar probes maintain the same detection time and sampling frequency, and simultaneously transmit detection signals to the fish. After receiving the echo signal from the fish, the signal is transmitted to the data processing module (server and edge computing module) for data processing.

[0042] In this embodiment, the data processing module is used to parse the raw sonar data of the sampling points into image data and depth measurement data, and to perform data processing.

[0043] In this embodiment, the data processing module is used to extract image data and depth measurement data from the raw sonar data, specifically including:

[0044] (S1) Image Data Processing

[0045] (S11) Each time the sonar transmits and receives a sonar pulse signal to the fish school, it is recorded as a Ping. The data processing module parses the raw data of each Ping into image data and depth measurement data. At the same time, the number of Pings M and the number of sampling points N in each Ping are combined into an M×N image. The sonar data of each Ping is converted into the corresponding gray value and stored in the image in sequence to obtain the original sonar image.

[0046] (S12) The original sonar image is sampled using the local median sampling method to reduce data redundancy.

[0047] Because sonar equipment has a high resolution, it has a large number of sampling points. The high resolution of sonar equipment leads to too many sampling points in each ping, resulting in redundant image information, excessive memory consumption, and slow computing speed. By sampling the sonar image based on local median sampling for each ping, the redundancy of image information caused by too many sampling points is reduced. At the same time, it avoids the disadvantages of poor correlation and poor robustness between data caused by equidistant sampling.

[0048] The method of local median sampling includes the following steps:

[0049] A 5×t local window is constructed on each Ping of the original sonar image, where t represents the sampling rate of the image in the column direction. This window is applied to the original two-dimensional sonar image, and the average value of the pixels within the entire window is calculated. The final average pixel value is used as the pixel of the sampled image. At the same time, the number of rows of the image is kept unchanged, and sampling is performed in the column direction of the image. The window moves with step sizes of t and 1 in the horizontal and vertical directions, respectively, to sample the entire image. For the first and last rows, the method of equal-interval sampling in the horizontal direction is directly used. Finally, the sampled sonar image is obtained.

[0050] (S2) The 3D point cloud fusion module is used to construct the 3D coordinates of each sampling point, and assign color information to each sampling point according to the gray value corresponding to each sampling point in the constructed 2D coordinate system, thereby constructing the 3D point cloud data of each sampling point.

[0051] (S21) Construct a three-dimensional coordinate system

[0052] The point cloud data uses the depth of each ping depth measurement data as the Z-axis, and the position of each ping depth measurement data center location as the X-axis and Y-axis, respectively.

[0053] The depth vector corresponding to each Ping depth measurement data is called Depth. The z-axis coordinate of each sampling point can be determined by the depth vector Depth. Depth is calculated using the following formula:

[0054] Depth=-range×sinθ×cos(pitch)-depth

[0055] Where pitch is the angle between the sonar probe and the XOZ plane, range is the slope range corresponding to each sampling point in the depth measurement data in each Ping data, θ is the angle between each sampling point in the depth measurement data in each Ping data and the XOY plane, and depth is the depth of the corresponding sonar probe from the water surface.

[0056] The X-axis (mid-x) and Y-axis (mid-y) represent the planar coordinates corresponding to the location of each ping depth data center. Considering the pitch angle, the slant distance is L.

[0057] X = sin(pitch) × L × sinα

[0058] Y = sin(pitch) × L × cosα

[0059] Where mid-x is the x-axis coordinate corresponding to the location of each ping depth sounding data center, mid-y is the y-axis coordinate corresponding to the location of each ping depth sounding data center, and α is the angle between the sonar probe and the YOZ plane.

[0060] After processing the depth sounding data as described above, we can obtain the three-dimensional coordinate information of each sampling point in the Ping depth sounding data.

[0061] (S22) The sampling points of image data and depth measurement data are not necessarily in a one-to-one correspondence in the horizontal distance direction. Image data has more sampling points, so they are more densely arranged than depth measurement data. To address this, interpolation is used to process the data, finding the corresponding color information for each depth measurement sampling point. For image data sampling points, which only have color information and horizontal distance, interpolation can be used to find the depth information corresponding to some sampling points. First, a two-dimensional coordinate system is established. The horizontal distance pos of each sampling point and its corresponding pixel value are projected onto the horizontal distance of each depth measurement point. The grayscale values ​​are then interpolated to obtain the approximate pixel information corresponding to that depth measurement sampling point, resulting in a better 3D display effect. The specific method is as follows:

[0062] Construction of a two-dimensional coordinate system:

[0063] Using the horizontal distance between image data and depth sounding data as the x-axis, the horizontal distance between image data and depth sounding data is calculated using the following formula:

[0064] pos = range × cosθ (1);

[0065] Where pos is the horizontal distance of each sampling point in the depth measurement data from the center position, range is the slope distance of each sampling point stored in the depth measurement data of each Ping data, and θ is the angle between each sampling point in the depth measurement data of each Ping data and the XOY plane.

[0066] A two-dimensional coordinate system is established with the grayscale value of the sampled sonar image as the vertical coordinate and the horizontal distance vector pos of each sampling point from the center position calculated by equation (1) as the horizontal coordinate.

[0067] Interpolation processing is performed on the depth sounding data in the established two-dimensional coordinate system.

[0068] Each Ping data point contains the corresponding depth information, which includes the slant range and angle for each sampling point. The sonar device's location is taken as the center, and its sampling frequency is fixed. It receives seabed echo signals at fixed time intervals from the start of sound wave transmission. Therefore, it is assumed that each sampling point is collected based on slant range. R represents the maximum slant range that the sonar device can detect, H is the height of the point from the seabed, and P represents the maximum horizontal distance. P is calculated using the following formula: The interval between each sampling point in the horizontal direction is P / ss, where ss represents the number of sampling points corresponding to the data acquisition module in the scanning area. The horizontal distance of each sonar image sampling point from the center position in each Ping is represented by the vector dis. The vector dis is used as the horizontal axis, and the corresponding vertical axis is the gray value of the sonar image corresponding to the sampling point. The three-dimensional point cloud color information is assigned, and the obtained points are further interpolated to the two-dimensional coordinates to increase the number of sampling points and improve the accuracy of the three-dimensional reconstruction of the fish swarm.

[0069] The horizontal axis represents the horizontal distance between sampling points, and the vertical axis represents the grayscale value corresponding to the sampling point. The sampling points are a series of discrete points on the coordinate system, and the coordinate axes of the corresponding depth sounding sampling points have the same horizontal distance. Then, the horizontal distance pos of the sampling point corresponding to each depth sounding data is projected, and the grayscale value is interpolated to obtain the approximate pixel information corresponding to the sampling point of the depth sounding data. The interpolation method used is cubic spline interpolation. The grayscale value of the sonar sampling point is used as a node. The node is composed of (x, y), where x represents the horizontal distance pos corresponding to the sampling point, and y represents the grayscale value corresponding to the sampling point. It is assumed that there are a total of n+1 nodes (x0, y0)...(xn, yn), where n is the number of sampling points, and S(x) represents the defined spline curve.

[0070] S i (x)=a i +b i (xx i )+c i (xx i ) 2 +d i (xx i ) 3 i = 0, 1, ..., n-1

[0071] Get a for each piecewise function i b i c i d i Ultimately, S was obtained.i The function (x) is used to interpolate the depth sampling points. The spline function curve obtained by this method is relatively smooth and can fit these discrete points well, which can better match reality. At the same time, the three-dimensional surface of the fish school is smoother.

[0072] (S23) Different sonar systems operate in different times and spaces. Due to errors in the sonar equipment itself or external interference in the seabed environment, the data measured by the sonar equipment may differ. This can lead to incomplete alignment of the overlapping parts between two sonar point clouds, affecting the accuracy and continuity of the overall fish school imaging. Therefore, by statistically analyzing the average distance from each point in the point cloud to a certain number of its nearest surrounding points, assuming that this average distance approximately follows a Gaussian distribution, points whose average distance is outside the standard range are removed. The specific method is as follows:

[0073] Based on the three-dimensional point cloud of the fish school processed in step (S22), point cloud registration is performed with an arbitrary three-dimensional point cloud as the main point cloud and the other three point clouds as auxiliary points cloud clouds. The point cloud registration method of ICP is used to find the transformation relationship between the two point clouds through rotation and translation transformation, so that the two point clouds are transformed to the same coordinate system, and a three-dimensional point cloud surface is obtained, thus constructing a local three-dimensional feature image of the fish school.

[0074] (S3) The three-dimensional point cloud data is converted into four-dimensional data using a three-dimensional reconstruction module, presenting the distribution of the fish school in three-dimensional space. In this embodiment, the local three-dimensional feature images of the fish school constructed by the four sonar probes are fused together to present the complete body shape features and underwater distribution of the fish school in three-dimensional space.

[0075] Based on the registered 3D point cloud surface, the maximum and minimum values ​​of the X and Y axes in the overlapping planes of the 3D point cloud surface are calculated to approximately determine the planar range of the 3D point cloud. Then, corresponding depth measurement data is added to it to form a preliminary 3D fish school graphic. Color information is then added to transform the original 3D space (x, y, z) into a 4D space (x, y, z, k). For points in the target point cloud, the closest point in the 3D space of the reference point cloud is selected as its corresponding point. At this point, only the 3D coordinate space position information of the point cloud is considered. However, the corresponding point found solely by spatial distance may not be accurate and may contain many incorrect corresponding points. Considering color information in addition to this allows for better alignment of the two point clouds. From the 3D curved point cloud data, the features extracted from the point cloud data at time t are matched one by one with the features extracted from the point cloud data at the same time t in the other three images. If a matching feature (the outline of the fish) is found in the image, a constraint relationship is established between the matching features. The similarity between the two image features is judged, and then the clearest image features in each frame are selected for registration. The value of keyframes in local images is measured and unnecessary keyframes are removed. If no matching feature is found in the image, it is added to the image to expand the scale of the feature map, making the reconstruction effect of the 3D scene more accurate and delicate, forming the final 3D distribution map of the fish school. This 3D distribution map includes the full 3D outline of individual fish and the underwater distribution state of the fish school.

[0076] This application uses multiple sonar probes to detect the distribution of underwater fish schools, and reconstructs the three-dimensional distribution of the fish schools by combining point cloud matching and feature matching in a three-dimensional point cloud fusion module. Point cloud matching can quickly form an initial three-dimensional distribution map of the fish schools without destroying the grayscale information of the image. On this basis, feature matching is used to construct a global, continuous, consistent, accurate and robust three-dimensional fish school map of a large-scale scene, reducing image distortion caused by insufficient detection.

[0077] The monitoring system of this application can understand the distribution and behavior of fish in the water through three-dimensional reconstruction of the fish population, thereby monitoring the fish population and assessing the health of the population. At the same time, it can calculate and predict the number of fish based on the imaging data.

[0078] The specific embodiments of this application have been described above. It should be understood that this application is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the invention. Unless otherwise specified, the embodiments and features described in these embodiments can be arbitrarily combined with each other.

[0079] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0080] The above provides a detailed description of a monitoring system and method for three-dimensional reconstruction of underwater fish schools provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A method for three-dimensional reconstruction of underwater fish school, characterized in that, The method comprises the following steps: (S1) The sonar transmitting unit in the data acquisition module transmits a detection signal, and the sonar receiving unit receives a return signal, and then transmits the signal to the data processing module; (S2) The data processing module extracts image data and depth data from the return signal, and performs data processing: combining the Ping number M and the sampling point number N in each Ping to form an M*N picture, converting each Ping sonar data into a corresponding gray value and storing it in the image to obtain an original sonar image, and performing sampling processing on the original sonar image by using a local median sampling method to reduce the redundancy of data; (S3) converting the sonar image data obtained in step (S2) to a two-dimensional coordinate system, taking the sampling points as the horizontal coordinates and the gray values as the vertical coordinates, establishing three-dimensional point cloud data through a three-dimensional point cloud fusion module, giving the three-dimensional point cloud data color information according to the established two-dimensional coordinate system, and performing point cloud registration and feature fusion on the three-dimensional point cloud data to obtain a local three-dimensional contour surface of the fish school; wherein the giving of the color information to the three-dimensional point cloud data according to the established two-dimensional coordinate system comprises: extracting the features of the point cloud data obtained by each sonar at the same time and performing feature matching between different sonar views one by one; if there are matching features, a constraint relationship is established between the matching features, the feature similarity is evaluated, the clearest image feature in each frame is selected for point cloud registration and feature fusion; if no matching features are found, the color information of the corresponding features is added to a feature map, and the size of the feature map is expanded. (S4) The three-dimensional local surface of the fish school established in step (S3) is spliced to obtain a complete three-dimensional contour of the fish individual and an underwater distribution map of the fish school.

2. The method of underwater fish school 3D reconstruction according to claim 1, characterized in that, The original sonar image is reduced in image information redundancy by the local median sampling method.

3. The method of underwater fish school 3D reconstruction according to claim 2, characterized in that, The local median sampling method comprises the following steps: scanning the extracted image data through a local window of 5*t, and calculating the mean value of the corresponding local window as the final image pixel value to obtain the sampled sonar image, wherein t is the sampling rate of the image in the column direction.

4. The method of underwater fish school 3D reconstruction of claim 1, wherein, The three-dimensional point cloud is established by the following method: The X-axis coordinate of the three-dimensional point cloud is: ; The Y-axis coordinate of the three-dimensional point cloud is: ; The Z-axis coordinate of the three-dimensional point cloud is: ; Wherein, pitch is the included angle between the sonar transmitting unit and XOZ plane, alpha is the included angle between the sonar transmitting unit and YOZ plane, range is the slant range corresponding to each sampling point in the corresponding depth data of each Ping data, theta is the included angle between each sampling point in the corresponding depth data of each Ping data and XOY plane, and depth is the depth of the corresponding sonar transmitting unit from the water surface.

5. The method of underwater fish school 3D reconstruction according to claim 4, characterized in that, Color information is given to the three-dimensional point cloud: a two-dimensional coordinate system is established with the horizontal distance of the sonar image sampling point as the horizontal coordinate and the gray value as the vertical coordinate, the depth information corresponding to the sampling point is found by interpolating the depth data based on the two-dimensional coordinate system, and the corresponding color information is assigned to each depth sampling point.

6. The monitoring system for three-dimensional reconstruction of underwater fish schools according to claim 5, characterized in that, The three-dimensional point cloud is processed as follows: the depth data and the image data are placed in the same coordinate system according to the horizontal distance and are subjected to cubic spline interpolation.

7. The method of underwater fish school 3D reconstruction according to claim 6, characterized in that, The cubic spline interpolation method comprises the following steps: The grayscale values ​​corresponding to the image data of the sonar sampling points are used as nodes. Each node consists of (x, y), and there are a total of n+1 nodes, represented as (x0, y0)...(x n y n ), Wherein, x represents the horizontal distance pos corresponding to the sampling point, y represents the gray value corresponding to the sampling point, represents the spline curve defined by the i th node, ​ The function of each segment is obtained a i ，b i ，c i ，d i and finally the function is obtained, from which the sampling points of the bathymetric data are interpolated.

8. The method of underwater fish school 3-D reconstruction of claim 1, wherein, The point cloud registration comprises point cloud fusion, which is used to construct a three-dimensional surface of the fish school.

9. The method of underwater fish school 3D reconstruction according to claim 8, characterized in that, The point cloud is fused by the following method: the image is converted by flipping and translating by using the ICP point cloud registration method to make the two point clouds in the same coordinate system, and the alignment between the point clouds is realized, and the overlapping data of the two point clouds is averaged.

Images