A kind of method and system for three-dimensional modeling of deep water net cage fish population based on sonar

Abstract

The present application relates to the technical field of data processing, and provides a kind of based on sonar's deep water net cage cultured fish school three-dimensional modeling method and system, by being arranged in the top geometric center position of target net cage multiple-beam sonar obtains global three-dimensional point cloud data, is arranged in multiple key depth layers of target net cage multiple multiple-beam sonar probe obtains local two-dimensional point cloud data, and according to the content information between global fish school three-dimensional point cloud data and each local fish school three-dimensional point cloud data based on the conversion of preset net cage topological coordinate system determines each fusion scheme, executes each fusion scheme and obtains target fish school three-dimensional point cloud data, and according to target fish school three-dimensional point cloud data carries out three-dimensional reconstruction and obtains fish school three-dimensional model and carries out visual display.The present application can not only guarantee the integrity of net cage water global space detection, but also can ensure the accuracy of local detail detection, effectively improve the reconstruction quality and visual display effect of fish school three-dimensional model.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method and system for three-dimensional modeling of deep-sea cage-cultured fish schools based on sonar. Background Technology

[0002] Deep-sea aquaculture is a semi-open, extensive aquaculture method that uses target cages for fish farming. It fully utilizes the superior water quality, large ecological capacity, and biodiversity of deep-sea environments, resulting in large-scale farming, rapid fish growth, and high-quality fish. Furthermore, it offers strong resistance to wind and waves and significant economic benefits. However, deep-sea aquaculture also presents challenges for aquaculture personnel in monitoring the activity and health of fish within the cages, enabling timely risk warnings and optimized feeding management.

[0003] Existing methods for underwater fish swarm detection and modeling are mainly divided into two categories: using sonar detection alone or combining sonar detection with underwater cameras or webcams to monitor fish swarm distribution. However, existing sonar detection methods do not address the limitations of sonar detection range and blind spots, failing to guarantee the accuracy and comprehensiveness of the detection data. Furthermore, underwater cameras or webcams, due to factors such as high operating conditions, limited shooting distance, and resolution, cannot truly provide reliable data support for monitoring fish swarm distribution. Consequently, it is impossible to construct a complete and reliable 3D model of fish swarms, making it even more difficult to achieve refined, scientific, and intelligent management of deep-sea aquaculture. Summary of the Invention

[0004] The purpose of this invention is to provide a sonar-based three-dimensional modeling method for deep-sea cage aquaculture fish schools. This method utilizes a multi-beam sonar array deployed at the geometric center of the cage top to acquire the overall spatial distribution of the water within the cage, and multi-beam sonar probes deployed at multiple key depth layers on the cage sidewalls to acquire local two-dimensional distributions at specific depth levels. This results in a complementary three-dimensional layered monitoring architecture combining a global overview and local side views. Combined with a reliable fusion mechanism of global three-dimensional point clouds and local high-resolution two-dimensional point clouds, this approach not only ensures the integrity of the global spatial detection of the cage water but also guarantees the accuracy of local detail detection, effectively improving the reconstruction quality and visualization effect of the three-dimensional fish school model.

[0005] To achieve the above objectives, it is necessary to provide a sonar-based three-dimensional modeling method and system for deep-sea cage aquaculture fish schools, addressing the aforementioned technical problems.

[0006] In a first aspect, embodiments of the present invention provide a method for three-dimensional modeling of deep-sea cage aquaculture fish schools based on sonar. A multibeam sonar array is deployed at the geometric center of the top of the target cage, and multiple multibeam sonar probes are deployed on the sides of multiple key depth layers of the target cage. The method includes: Global three-dimensional point cloud data is acquired based on the multibeam sonar, and corresponding local two-dimensional point cloud data is acquired based on each of the multibeam sonar probes. Based on the preset cage topology coordinate system, the global three-dimensional point cloud data is converted into the corresponding global fish swarm three-dimensional point cloud data, and each group of local two-dimensional point cloud data is converted into the corresponding local fish swarm three-dimensional point cloud data. Based on the content information between the global fish swarm 3D point cloud data and each of the local fish swarm 3D point cloud data, each fusion scheme is determined. Each of the aforementioned fusion schemes is executed to obtain three-dimensional point cloud data of the target fish swarm; wherein, each of the aforementioned fusion schemes includes single-parameter controlled fusion for a first region and dual-parameter collaborative fusion for a second region, the first region being determined based on the content information, and the second region being a region other than the first region; A 3D reconstruction is performed based on the 3D point cloud data of the target fish school to obtain a corresponding 3D model of the fish school, and the 3D model of the fish school is then visualized.

[0007] Furthermore, the global three-dimensional point cloud data includes the Cartesian coordinates and echo intensity of different detection points; The steps for acquiring global three-dimensional point cloud data based on the multibeam sonar include: According to the preset detection frequency, the first raw beam data is collected by the multi-beam sonar. Based on the marine environmental monitoring data at the sampling time corresponding to the first original beam data, the actual sound ray path is obtained by performing sound ray bending correction based on the ray tracing method; the marine environmental monitoring data is obtained based on environmental sensor arrays deployed at various key depth layers; the environmental sensor arrays include temperature sensors, salinity sensors and pressure sensors. Based on the actual sound path, the slant range, horizontal angle, and pitch angle in the first original beam data are corrected to obtain the first corrected beam data. The first corrected beam data is transformed into Cartesian coordinates to obtain the global three-dimensional point cloud data.

[0008] Furthermore, the marine environmental monitoring data includes temperature, salinity, and depth at various key depth layers; The step of obtaining the actual sound ray path by performing sound ray bending correction based on the marine environmental monitoring data corresponding to the sampling time of the first original beam data includes: Based on the temperature, salinity, and depth of each key depth layer, the actual sound speed of the corresponding key depth layer is calculated using a preset classical ocean sound speed model. Based on the actual sound velocity of each of the key depth layers, a sound velocity profile is constructed using a preset interpolation algorithm. Based on the sound velocity profile, the actual sound ray path is obtained by performing sound ray bending correction using the ray tracing method.

[0009] Furthermore, the global three-dimensional point cloud data includes the Cartesian coordinates and echo intensity of different detection points; The step of converting the global 3D point cloud data into corresponding global fish swarm 3D point cloud data based on a preset cage topology coordinate system includes: The pre-calibrated sonar installation deflection angle is summed with the component data of the corresponding acquired cage attitude angle to calculate the comprehensive rotation angle; the sonar installation deflection angle includes lateral deflection angle and longitudinal deflection angle. Based on the comprehensive rotation angle, a first rotation matrix is ​​constructed, and attitude compensation is performed on the global three-dimensional point cloud data according to the first rotation matrix to obtain attitude-compensated three-dimensional point cloud data. Based on the rotation angle of the main symmetry axis of the cage, the attitude compensation three-dimensional point cloud data is corrected by the cage rotation angle to obtain rotation angle corrected three-dimensional point cloud data; the rotation angle of the main symmetry axis of the cage is calculated based on multiple acoustic beacons deployed on the rigid structure of the target cage; Based on a preset echo intensity range, background noise is removed from the rotation angle-corrected 3D point cloud data to obtain the global fish swarm 3D point cloud data.

[0010] Furthermore, the local two-dimensional point cloud data includes the horizontal plane coordinates and echo intensity of each detection point within different key depth layers; Based on a preset cage topology coordinate system, the steps for converting the local two-dimensional point cloud data of each group into corresponding local three-dimensional point cloud data of the fish school include: Based on the installation depth and beam vertical opening angle of each of the multibeam sonar probes, the corresponding local two-dimensional point cloud data is stretched in the depth direction to obtain the corresponding pseudo local three-dimensional point cloud data. Based on the calibration position and probe attitude angle of each of the multibeam sonar probes in the preset cage topology coordinate system, the corresponding coordinate transformation matrix is ​​obtained; Based on the coordinate transformation matrix corresponding to the multibeam sonar probe, the corresponding pseudo-local 3D point cloud data is transformed to obtain the corresponding candidate local 3D point cloud data. Based on a preset echo intensity range, background noise is removed from the candidate local 3D point cloud data to obtain the local fish swarm 3D point cloud data.

[0011] Furthermore, the step of stretching the corresponding local two-dimensional point cloud data in the depth direction based on the installation depth and beam vertical opening angle of each of the multi-beam sonar probes to obtain the corresponding pseudo-local three-dimensional point cloud data includes: Based on the vertical beam opening angle and installation depth of each of the multibeam sonar probes, the corresponding total coverage range in the vertical direction of the beam is obtained based on the geometric relationship between the beam centerline and the coverage radius. Based on the total vertical coverage of each multibeam sonar probe and the installation depth, the corresponding effective depth coverage is obtained. Based on the effective depth coverage range and preset depth change step size of each of the multibeam sonar probes, the depth direction coordinates of each two-dimensional plane coordinate in the corresponding local two-dimensional point cloud data are serialized and expanded to generate the corresponding pseudo local three-dimensional point cloud data.

[0012] Furthermore, the content information includes at least the effective coverage area. The determination of various fusion schemes based on the content information between the global fish swarm 3D point cloud data and each of the local fish swarm 3D point cloud data includes: Each point cloud depth layer is obtained based on the installation depth of each multibeam sonar probe and the effective coverage range of the depth. Based on each point cloud depth layer, the global fish swarm 3D point cloud data is divided into point cloud data of each depth layer; Determine the fusion scheme for fusing the depth layer point cloud data with the corresponding local fish swarm 3D point cloud data.

[0013] Furthermore, the content information also includes the morphological characteristics of the fish school; The execution of each of the aforementioned fusion schemes yields 3D point cloud data of the target fish swarm, including: During the execution of each fusion scheme, the first region and the second region are determined based on the fish morphological characteristics. First data corresponding to the first region and second data corresponding to the second region are extracted from the deep layer point cloud data, and third data corresponding to the first region and fourth data corresponding to the second region are extracted from the local fish swarm 3D point cloud data. Based on the data in the local fish swarm 3D point cloud data, the first data and the third data are fused, and the second data and the fourth data are weighted and fused. By integrating the point cloud fusion results after each of the aforementioned fusion schemes is executed, the three-dimensional point cloud data of the target fish group is obtained.

[0014] Furthermore, the step of performing 3D reconstruction based on the target fish school's 3D point cloud data to obtain a corresponding 3D model of the fish school, and then visualizing the 3D model of the fish school, includes: The target fish group's 3D point cloud data is preprocessed to obtain the 3D point cloud data to be analyzed; the preprocessing includes background filtering, outlier removal, and normalization. Based on the three-dimensional point cloud data to be analyzed, the corresponding fish individual point cloud is obtained based on a preset point cloud segmentation deep learning model. The current static features of each individual fish point cloud are obtained; the current static features of the fish include the fish position, geometric features, pose features and point cloud descriptors; The historical static features of the fish body at the previous few sampling times corresponding to each individual fish point cloud are combined with the current static features of the fish body in a temporal sequence to obtain the corresponding temporal features of the individual fish point cloud. Based on the temporal features of the individual fish point clouds corresponding to each individual fish point cloud, behavior recognition is performed based on a pre-built behavior classification model to obtain the corresponding behavior category. Based on the Poisson reconstruction algorithm, the surface of the point cloud of each individual fish is reconstructed, and the three-dimensional model of the fish group is obtained according to the obtained three-dimensional model of the individual fish and the corresponding fish body position. Based on the current static features and behavioral categories of each individual fish point cloud, the 3D model of the fish school is rendered and displayed.

[0015] Secondly, embodiments of the present invention provide a sonar-based three-dimensional modeling system for deep-sea cage aquaculture fish schools. A multi-beam sonar array is deployed at the geometric center of the top of the target cage, and multiple multi-beam sonar probes are deployed on the sides of multiple key depth layers of the target cage. The system includes: The data acquisition module is used to acquire global three-dimensional point cloud data based on the multibeam sonar, and to acquire corresponding local two-dimensional point cloud data based on each of the multibeam sonar probes. The point cloud conversion module is used to convert the global three-dimensional point cloud data into the corresponding global fish swarm three-dimensional point cloud data based on the preset cage topology coordinate system, and to convert each group of local two-dimensional point cloud data into the corresponding local fish swarm three-dimensional point cloud data. The fusion scheme generation module is used to determine each fusion scheme based on the content information between the global fish swarm 3D point cloud data and each of the local fish swarm 3D point cloud data. The point cloud fusion module is used to execute each of the fusion schemes to obtain the three-dimensional point cloud data of the target fish swarm; wherein, each of the fusion schemes includes single-parameter controlled fusion for a first region and dual-parameter collaborative fusion for a second region, the first region being determined based on the content information, and the second region being a region other than the first region; The 3D reconstruction module is used to perform 3D reconstruction based on the 3D point cloud data of the target fish group to obtain the corresponding 3D model of the fish group, and to visualize the 3D model of the fish group.

[0016] This invention provides a sonar-based method and system for three-dimensional modeling of fish schools in deep-sea cage aquaculture. Compared with existing technologies, this sonar-based method provides a global three-dimensional overview of the entire cage water space by deploying multibeam sonar at the geometric center of the cage top. It also provides a three-dimensional cross-monitoring architecture by deploying multiple multibeam sonar probes at key depth layers on the cage sidewalls, offering a combination of high-resolution, close-range two-dimensional scanning from a global top-down perspective and high-precision local side-view perspectives. Combined with a reliable fusion mechanism of global three-dimensional point clouds and local high-resolution two-dimensional point clouds, this method not only ensures the integrity of the global spatial detection of the cage water body but also guarantees the refinement of local detail detection. This effectively improves the reconstruction quality and visualization effect of the three-dimensional fish school model, providing a reliable analytical basis for accurately assessing the fish population, size distribution, behavior, health status, and cage safety. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the three-dimensional modeling method for deep-sea cage culture fish based on sonar in an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of the sonar-based three-dimensional modeling system for deep-sea cage aquaculture fish schools in an embodiment of the present invention; The attached figures are labeled as follows: 1. Data acquisition module; 2. Point cloud conversion module; 3. Fusion scheme generation module; 4. Point cloud fusion module; 5. 3D reconstruction module. Detailed Implementation

[0018] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the embodiments described below are only part of the embodiments of this invention and are used to illustrate the invention, but are not intended to limit the scope of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0019] It should be noted in advance that in existing deep-sea cage aquaculture, after the cage is set up, a data acquisition device is usually installed on the top of the cage for subsequent 3D model construction. However, in the actual 3D modeling process, the number of pixels, angles, and coordinates in the modeling data will affect the final 3D modeling effect. Relying solely on a data acquisition device on the top is simply not enough to construct an accurate 3D model, which will have a very adverse impact on the subsequent fish farming control.

[0020] In one embodiment of the present invention, such as Figure 1 As shown, a sonar-based 3D modeling method for deep-sea cage aquaculture fish schools is provided. In terms of data sensors, it is no longer limited to the top location and a single number, but instead deploys a multi-beam sonar at the geometric center of the top of the target cage, and multiple multi-beam sonar probes are deployed on the sides of the target cage at several key depth layers. The target cage can be cylindrical or square, etc. Both the multi-beam sonar and the probes use existing equipment, and the key depth layers can be set according to actual monitoring application requirements and the detection coverage of the multi-beam sonar probes. For example, a multi-beam sonar probe can be deployed at 1 / 4, 2 / 4, and 3 / 4 of the cage depth, respectively; no specific limitation is made here.

[0021] The multibeam sonar deployed at the top in this embodiment provides a global three-dimensional overview of the entire net cage water space, overcoming the blind spot problem that cannot be effectively observed in the top and central areas of the net cage when multibeam sonar probes are only deployed on the sides. Multibeam sonar probes deployed at various key depth layers provide high-resolution, close-range two-dimensional scans at specific depth levels, capturing more detailed information such as fish distribution, size, and behavior within that layer. This compensates for the shortcomings of the top sonar in local detail and vertical resolution, actively and structurally combining the global overhead view with the high-precision side view to form a three-dimensional cross-monitoring network, achieving data complementarity at different levels and with varying precision. Specifically, the method for three-dimensional modeling of deep-water net cage aquaculture fish schools based on the aforementioned three-dimensional cross-monitoring network of top multibeam sonar and multibeam sonar probes deployed at multiple key depth layers on the sides includes: S11. Obtain global three-dimensional point cloud data based on the multibeam sonar, and obtain corresponding local two-dimensional point cloud data based on each of the multibeam sonar probes; In this embodiment, to ensure the synchronous acquisition of global 3D point cloud data and local 2D point cloud data, the frequency at which the multibeam sonar and multibeam sonar probe send acoustic pulses and acquire beam data for each corresponding detection point is set to the same detection frequency. The specific value of the detection frequency can be adjusted based on actual monitoring needs and is not specifically limited. In practical applications, a central control unit can be set up to globally and synchronously trigger the multibeam sonar and multibeam sonar probe to perform detection tasks according to the detection frequency. The multibeam sonar and multibeam sonar probe are synchronized with a high-precision, low-drift common clock source (such as setting up a dedicated synchronization signal generator) to ensure time alignment. The specific design can be implemented with reference to relevant existing technologies, which will not be detailed here.

[0022] In this embodiment, the global three-dimensional point cloud data includes the Cartesian coordinates and echo intensity of different detection points; specifically, the step of acquiring global three-dimensional point cloud data based on the multibeam sonar includes: According to the preset detection frequency, the first raw beam data is acquired by the multi-beam sonar. The acquisition of the first raw beam data can be obtained by referring to the beam vector data acquisition method of existing multi-beam sonar. The corresponding first raw beam data includes the slant range (the distance between the detection points calculated based on the round-trip time of the sound wave), horizontal angle (the pointing angle of the beam in the horizontal plane, usually referring to the azimuth angle in the transducer coordinate system), pitch angle (the pointing angle of the beam in the vertical plane, usually referring to the angle between the beam and the vertical direction or the horizontal plane), and echo intensity (the amplitude or energy of the received signal), which is given based on the assumption of constant sound speed (straight-line propagation).

[0023] Based on the marine environmental monitoring data at the sampling time corresponding to the first original beam data, sound ray bending correction is performed using the ray tracing method to obtain the actual sound ray path. The marine environmental monitoring data can be understood as taking into account the differences in water temperature and salinity at different depths in the ocean, and how these differences affect sound speed. This data is collected by an environmental sensor array deployed at various key depth layers of the target cage. The environmental sensor array includes temperature sensors, salinity sensors, and pressure sensors, and the corresponding marine environmental monitoring data includes the temperature, salinity, and depth at each key depth layer. Specifically, the step of obtaining the actual sound ray path by performing sound ray bending correction based on the marine environmental monitoring data at the sampling time corresponding to the first original beam data includes: Based on the temperature, salinity, and depth of each key depth layer, the actual sound speed of the corresponding key depth layer is calculated using a preset classical ocean sound speed model. The preset classical ocean sound speed model can be understood as an existing calculation expression for the sound speed of seawater propagation based on temperature, salinity, and depth, such as the Mackenzie empirical formula for seawater sound speed and the Medwin empirical formula for seawater sound speed, etc., which are not limited here.

[0024] Based on the actual sound velocity of each key depth layer, a sound velocity profile is constructed using a preset interpolation algorithm (such as linear interpolation or spline interpolation). The construction process of the sound velocity profile (SSP) can be understood as using the actual sound velocity at discrete key depth layers to generate a continuous function describing the change of sound velocity with depth through an interpolation algorithm. The preset interpolation algorithm can be implemented using linear interpolation or spline interpolation. Based on a preset resolution (depth interval), the interpolation algorithm calculates the sound velocity values ​​at a series of discrete depth points between two adjacent key depth layers and the actual sound velocity data pairs to obtain a continuous sound velocity-depth relationship function (sound velocity profile).

[0025] Based on the sound velocity profile, the actual sound ray path is obtained by performing sound ray bending correction using the ray tracing method. The process of obtaining the actual sound ray path is as follows: the sound velocity profile is divided into multiple depth layers where the sound velocity can be approximated as constant or linearly related to the depth; based on the beam elevation angle when the sound ray exits the transducer and enters the first profile layer, and the depth and sound velocity of the adjacent next profile layer, the refraction angle of the beam entering the next profile layer is calculated based on the acoustic Snell's law; then, based on the sound velocity, angle, and layer thickness of the profile layer, the propagation path segment of the sound ray within that layer is calculated based on geometric relationships; and so on, the propagation path segments of all profile layers can be obtained; connecting the sound ray path segments (small straight line segments) calculated layer by layer starting from the transducer yields the entire curved sound ray path from the transducer to the detection point, i.e., the required actual sound ray path.

[0026] Based on the actual sound path, the slant range, horizontal angle, and pitch angle in the first original beam data are corrected to obtain the first corrected beam data. The corrected slant range in the first corrected beam data is the arc length of the original slant range as the actual sound path. The corrected angles (horizontal angle and pitch angle) need to be recalculated based on the tangent direction of the actual sound path at the transducer. The corrected angles reflect the true direction of the sound ray when it leaves the transducer. That is, the corrected first corrected beam data contains parameters that more accurately describe the actual propagation path of the sound wave and the position of the detection point.

[0027] The first corrected beam data is transformed into Cartesian coordinates to obtain the global three-dimensional point cloud data. The azimuth, elevation, and slant range in the first corrected beam data can be understood as spherical coordinates with the transducer as the origin, and the Cartesian coordinate system can be understood as the northeast-sky coordinate system. Based on the existing coordinate transformation formula between spherical and Cartesian coordinates, the Cartesian coordinates of each detection point can be easily obtained, while retaining the echo intensity of each detection point in the first corrected beam data. Combining the Cartesian coordinates and echo intensities of all detection points forms the global three-dimensional point cloud data describing all detection points.

[0028] In this embodiment, each set of local two-dimensional point cloud data includes the horizontal two-dimensional coordinates (Cartesian coordinates without a Z-axis) and echo intensity of each detection point within different key depth layers. The specific steps for acquiring the corresponding local two-dimensional point cloud data based on each multibeam sonar probe include: acquiring the corresponding second raw beam data through each multibeam sonar probe according to a preset detection frequency; the second raw beam data includes the azimuth angle (elevation angle of 0 degrees) and slant range of each beam; based on the marine environmental monitoring data at the sampling time corresponding to the second raw beam data, performing acoustic ray bending correction using the ray tracing method to obtain the corresponding second corrected beam data; converting each set of second corrected beam data into local two-dimensional point cloud data in a Cartesian coordinate system; it should be noted that the acquisition process of the second corrected beam data here can refer to the aforementioned acquisition process of the first corrected beam data, and will not be detailed here.

[0029] This embodiment considers the influence of temperature, depth, and salinity on the sound speed of seawater. It uses top multibeam sonar to obtain reliable global three-dimensional point cloud data to represent the overall spatial distribution, and uses multibeam sonar probes at each key depth layer to obtain high-precision two-dimensional point cloud data to represent the distribution information in the local scanning plane. This provides a reliable data foundation for subsequent fusion analysis to obtain comprehensive and high-precision three-dimensional point cloud data of fish schools.

[0030] S12. Based on the preset cage topology coordinate system, the global three-dimensional point cloud data is converted into the corresponding global fish swarm three-dimensional point cloud data, and each group of local two-dimensional point cloud data is converted into the corresponding local fish swarm three-dimensional point cloud data; wherein, the origin of the preset cage topology coordinate system is the top geometric center of the target cage; the Z-axis of the preset cage topology coordinate system is the direction of gravity, and the X-axis and Y-axis are two orthogonal axes in the horizontal plane automatically determined based on the principal symmetry axis of the target cage.

[0031] In this embodiment, the global fish swarm 3D point cloud data can be understood as a fish swarm 3D point cloud in a preset cage topology coordinate system obtained by processing the global 3D point cloud data; specifically, the step of converting the global 3D point cloud data into the corresponding global fish swarm 3D point cloud data based on the preset cage topology coordinate system includes: The pre-calibrated sonar installation deflection angle is summed with the corresponding acquired cage attitude angle component data to calculate the comprehensive rotation angle; the sonar installation deflection angle includes the lateral deflection angle and the longitudinal deflection angle; wherein, the cage attitude angle can be understood as the roll angle and pitch angle obtained based on the attitude sensor deployed on the target cage; the cage attitude angle and the sonar installation deflection angle are defined on the same rotation axis, that is, in the obtained comprehensive rotation angle, the total rotation angle around the Y-axis is the sum of the lateral deflection angle and the roll angle, and the total rotation angle around the X-axis is the sum of the longitudinal deflection angle and the pitch angle.

[0032] Based on the comprehensive rotation angle, a first rotation matrix is ​​constructed, and attitude compensation is performed on the global 3D point cloud data according to the first rotation matrix to obtain attitude-compensated 3D point cloud data. The construction process of the first rotation matrix may include obtaining a Y-axis rotation matrix based on the total rotation angle around the Y-axis in the comprehensive rotation angle, and obtaining an X-axis rotation matrix based on the total rotation angle around the X-axis in the comprehensive rotation angle, according to the X-axis rotation principle. Then, the X-axis rotation matrix is ​​multiplied left by the Y-axis rotation matrix in the order of first Y-axis rotation and then X-axis rotation to obtain the required first rotation matrix. After obtaining the first rotation matrix, the 3D coordinates of each detection point in the global 3D point cloud data are transformed using the first rotation matrix to obtain the required attitude-compensated 3D point cloud data.

[0033] Based on the rotation angle of the main symmetry axis of the cage, the attitude compensation three-dimensional point cloud data is corrected by the cage rotation angle to obtain rotation angle corrected three-dimensional point cloud data. The cage rotation angle correction can be understood as taking into account the influence of water flow on the cage, and to avoid directional deviations in the point cloud data caused by the overall rotation of the cage, ensuring that point cloud data acquired at different times and from different perspectives have a consistent directional reference. The main symmetry axis rotation angle used for correction is calculated based on multiple acoustic beacons deployed on the rigid structure of the target cage. To ensure the accuracy of the main symmetry axis rotation angle acquisition, at least three acoustic beacons are used. The reference three-dimensional coordinates of each acoustic beacon in the preset cage topology coordinate system are pre-calibrated. The specific installation positions of the acoustic beacons can vary depending on the shape of the cage, as long as they meet the positioning application requirements. For example, for a square cage, one acoustic beacon can be installed at each of the four corner points. Assuming four acoustic beacons are deployed inside the cage, the process for obtaining the cage's principal axis of symmetry rotation angle is as follows: Calculate the reference attitude centroid coordinates based on the reference 3D coordinates of the four acoustic beacons, and calculate the current attitude centroid coordinates based on the current 3D coordinates of the four acoustic beacons in the attitude-compensated 3D point cloud data to eliminate translation effects and retain only the rotation component; subtract each reference 3D coordinate from the reference attitude centroid coordinates to obtain the corresponding decentralized reference 3D coordinates, and subtract each current 3D coordinate from the current attitude centroid coordinates to obtain the corresponding decentralized current 3D coordinates; obtain the covariance matrix by summing the dot products of the decentralized reference 3D coordinate vectors of each acoustic beacon and the corresponding decentralized current 3D coordinate vectors, and perform singular value decomposition on the covariance matrix to obtain the left and right singular vector matrices; obtain the target rotation matrix by multiplying the transposes of the left and right singular vector matrices, and then solve for the required cage principal axis of symmetry rotation angle based on the relationship between the target rotation matrix and the cage principal axis of symmetry rotation angle. Alternatively, one can directly use the target rotation matrix obtained to correct the rotation angle of the cage's principal axis of symmetry without calculating the cage's rotation angle, thus obtaining the rotation angle-corrected 3D point cloud data.

[0034] Based on a preset echo intensity range, background noise is removed from the rotation angle-corrected 3D point cloud data to obtain the global fish swarm 3D point cloud data. The preset echo intensity range can be set according to the difference between the echo intensity range of fish in the actual application scenario and the echo intensity range of various parts of the net cage body, and is not specifically limited here. That is, the global fish swarm 3D point cloud data is obtained by removing data points in the rotation angle-corrected 3D point cloud data whose echo intensity is not within the preset echo intensity range.

[0035] In this embodiment, dynamic compensation is achieved by fusing the sonar installation angle and the real-time attitude of the cage. This is combined with correction by calculating the rotation angle of the principal axis of symmetry using acoustic beacons on the cage structure, and a fish swarm point cloud acquisition mechanism that effectively eliminates background noise using a preset echo intensity range. This significantly improves the spatial positioning accuracy and stability of the point cloud, providing reliable data support for the accuracy of subsequent three-dimensional monitoring and analysis of the fish swarm.

[0036] In this embodiment, the local two-dimensional point cloud data includes the horizontal plane coordinates and echo intensity of each detection point within different key depth layers; the steps for converting each set of local two-dimensional point cloud data into corresponding local fish swarm three-dimensional point cloud data based on a preset cage topology coordinate system include: Based on the installation depth and beam vertical opening angle of each of the multi-beam sonar probes, the corresponding local two-dimensional point cloud data is stretched in the depth direction to obtain the corresponding pseudo-local three-dimensional point cloud data. Here, the installation depth can be understood as the depth position of the cage where the multi-beam sonar probe is installed, and the beam vertical opening angle is also the beam vertical opening angle. To facilitate the subsequent use of high-resolution data from the local two-dimensional point cloud data to effectively supplement the global three-dimensional point cloud data, this embodiment preferably involves stretching the local two-dimensional point cloud data corresponding to each key depth layer in the depth direction to generate the corresponding three-dimensional point cloud data. Specifically, the step of stretching the corresponding local two-dimensional point cloud data in the depth direction based on the installation depth and beam vertical opening angle of each of the multi-beam sonar probes to obtain the corresponding pseudo-local three-dimensional point cloud data includes: Based on the vertical beam opening angle and installation depth of each multi-beam sonar probe, the corresponding total vertical beam coverage is obtained based on the geometric relationship between the beam centerline and the coverage radius. That is, the total vertical beam coverage is equal to twice the product of the tangent of the half-angle corresponding to the vertical beam opening angle and the installation depth, which can be expressed as: w=2htan(θ / 2), where h represents the installation depth, θ represents the vertical beam opening angle, and w represents the total vertical beam coverage. Specifically, it can be calculated by referring to relevant similar triangle methods, coordinate methods, vector methods, area methods, and differential methods, etc., which will not be detailed here.

[0037] Based on the total vertical coverage of each multibeam sonar probe and the installation depth, the corresponding effective depth coverage is obtained. Specifically, the effective depth coverage is obtained by taking half of the total vertical coverage as the maximum depth offset relative to the installation depth, obtaining the lower limit of the depth coordinate range based on the difference between the installation depth and the maximum depth offset, and obtaining the upper limit of the depth coordinate range based on the sum of the installation depth and the maximum depth offset.

[0038] Based on the effective depth coverage range and preset depth variation step size of each of the multibeam sonar probes, the depth direction coordinates of each two-dimensional plane coordinate in the corresponding local two-dimensional point cloud data are serialized and expanded to generate corresponding pseudo-local three-dimensional point cloud data. The process of serializing and expanding the depth direction coordinates of each two-dimensional plane coordinate can be understood as follows: determining the corresponding depth Z-coordinate value range based on the corresponding effective depth coverage range; traversing from the lower limit to the upper limit of the depth Z-coordinate value range according to the preset depth variation step size to generate a depth direction coordinate sequence (Z-coordinate value sequence); then supplementing the obtained depth direction coordinate sequence with the Z-axis coordinates of each two-dimensional plane coordinate to obtain the corresponding three-dimensional coordinates. Through the above expansion method, and so on, the pseudo-local three-dimensional point cloud data corresponding to each set of local two-dimensional point cloud data can be obtained. The generation of pseudo-local three-dimensional point cloud data can more realistically reflect the coverage range of sound waves in the vertical direction, providing a point cloud base data that is closer to the actual three-dimensional distribution for subsequent processing, significantly improving the accuracy of the spatial dimension information of the original data.

[0039] Based on the calibration positions and probe attitude angles of each multibeam sonar probe in the preset cage topology coordinate system, coordinate system transformation is performed on the corresponding pseudo-local 3D point cloud data to obtain corresponding candidate local 3D point cloud data. The candidate local 3D point cloud data can be understood as the 3D point cloud obtained by rotating and translating the point cloud of each multibeam sonar probe from its own local coordinate system to the global cage topology coordinate system. The acquisition process involves first determining the calibration positions of each multibeam sonar probe in the preset cage topology coordinate system (probe center in the cage topology coordinate system). The coordinate transformation matrix (including the rotation matrix and translation vector) is obtained by taking the probe's coordinates in the cage coordinate system and the probe's attitude angle (the rotation angle of the probe relative to the cage coordinate system, including the rotation angle around the X-axis, Y-axis, and Z-axis). Then, the pseudo-local 3D point cloud data is transformed according to the coordinate transformation matrix to obtain the final coordinates. It should be noted that the rotation matrix in the coordinate transformation matrix is ​​based on the probe's attitude angle and is constructed in the order of first selecting the Z-axis, then rotating the Y-axis, and finally rotating the X-axis. The translation vector is the coordinate vector of the probe center in the cage coordinate system. By fully considering the actual installation orientation and possible slight tilt of the sonar probe in the complex cage environment, the local coordinate system of each probe is aligned to the preset global cage topology coordinate system, greatly eliminating positioning errors caused by probe installation deviations and attitude changes, ensuring that points detected by different probes can be accurately fused and compared in the same 3D space.

[0040] Based on a preset echo intensity range, background noise is removed from the candidate local 3D point cloud data to obtain the local fish swarm 3D point cloud data; that is, the local fish swarm 3D point cloud data is the number of points obtained by removing data points in the candidate local 3D point cloud data whose echo intensity is outside the preset echo intensity range. This embodiment considers the presence of strong echoes from numerous fixed structures (nets, ropes, frames) in the net cage environment, as well as weak noise from suspended matter and bubbles in the water. Utilizing the characteristic that fish echo intensity is usually between strong structural echoes and weak environmental noise, a preset or adaptively set intensity range is used to efficiently remove strong reflection interference and low-intensity background noise from the net cage structure. This results in the "local fish swarm 3D point cloud" primarily containing the target fish swarm signal, significantly improving the signal-to-noise ratio and providing a more reliable foundation for subsequent fish swarm state analysis (such as density estimation, size identification, and behavior tracking).

[0041] This embodiment introduces a custom coordinate system closely related to the physical structure of the target cage, rather than relying solely on the geodetic coordinate system or the device's own coordinate system. This system is used to transform the global 3D point cloud and all local 2D point clouds into this unified cage topology coordinate system, naturally solving the spatial alignment problem of different sensor data and providing reliable technical support for fusing point cloud data from different sources, locations, and dimensions.

[0042] S13. Based on the content information between the global fish swarm 3D point cloud data and each of the local fish swarm 3D point cloud data, determine each fusion scheme; wherein, the content information includes at least the effective depth coverage range, and the effective depth coverage range can be understood as the point cloud depth direction coordinate range corresponding to the local fish swarm 3D point cloud data.

[0043] Specifically, determining each fusion scheme based on the content information between the global fish swarm 3D point cloud data and each of the local fish swarm 3D point cloud data includes: Each point cloud depth layer is obtained based on the installation depth of each multibeam sonar probe and the effective depth coverage range; wherein, the number of point cloud depth layers is the same as the number of multibeam sonar probes, and the thickness of each point cloud depth layer is the effective depth coverage range of the corresponding probe.

[0044] Based on each point cloud depth layer, the global fish swarm 3D point cloud data is divided into point cloud data of various depth layers. The process of dividing the global fish swarm 3D point cloud data into multiple depth layers can be understood as allocating the Z-axis coordinate (depth value) in the global fish swarm 3D point cloud data to the point cloud depth layer with the corresponding depth range based on the depth range of each point cloud depth layer. The point cloud data contained in each point cloud depth layer is the depth layer point cloud data.

[0045] A fusion scheme is determined for fusing the deep layer point cloud data with the corresponding local fish swarm 3D point cloud data; wherein, the fusion scheme can be understood as a partitioned fusion scheme that determines different fusion methods for different density areas based on the data distribution in the deep layer point cloud data and the corresponding local fish swarm 3D point cloud data, so as to obtain point cloud data that simultaneously takes into account the requirements of comprehensiveness and high accuracy, and makes better use of subsequent high-quality modeling and analysis.

[0046] This embodiment is based on a layered fusion design of global point cloud slicing and local point cloud, which can effectively ensure that the fusion of point cloud data at different depths does not interfere with each other and is efficient, and can improve the reliability of the fused point cloud data to a certain extent.

[0047] S14. Execute each of the aforementioned fusion schemes to obtain the three-dimensional point cloud data of the target fish swarm; wherein, each of the aforementioned fusion schemes includes single-parameter control fusion for the first region and dual-parameter collaborative fusion for the second region, the first region being determined based on the content information, and the second region being the region other than the first region.

[0048] In this embodiment, the deep layer point cloud data and local fish swarm 3D point cloud data fused in each fusion scheme need to be divided into two regions (a first region and a second region), and different standards are selected for fusion. This allows for the full utilization of all available data to achieve complementary advantages based on the different characteristics of the two types of data, thereby ensuring the overall coverage integrity, high quality, and high accuracy of the fused point cloud data. Specifically, the first region can be understood as the fish gathering area, which needs to reflect accurate and clear fish swarm details. This is difficult to obtain through multibeam sonar on the top of the net cage. At this time, a single parameter needs to be selected for selective control of fusion, that is, the data content of the local fish swarm 3D point cloud data needs to be used as the standard. In this embodiment, the first region reflecting the fish swarm details is determined by extracting the fish morphological features from the global fish swarm 3D point cloud data and the local fish swarm 3D point cloud data. Considering that the fish gathering area will have obvious regional edge undulations, it can be accurately extracted using existing point cloud convex hull or principal component analysis techniques. For example, in practical applications, the convex hull technique of point cloud can be used to calculate the convex hull of the local fish swarm 3D point cloud data on the horizontal plane to quickly determine the location of the first region. Then, the locations other than the first region are all second regions that reflect information about non-fish-gathering areas (which can be understood as areas where fish are scattered).

[0049] In the above embodiments, the first region, identified as a fish-gathering area, is a key analysis location for subsequent 3D reconstruction. Accurate and high-precision data must be ensured during fusion. The second region, a non-fish-gathering area, requires comprehensive data while maximizing accuracy during fusion; that is, different regions need to be fused according to different standards. Specifically, the content information also includes fish morphological features (point cloud gathering area outline). The execution of each fusion scheme to obtain the target fish swarm 3D point cloud data includes: During the execution of each fusion scheme, the first region and the second region are determined based on the fish morphological characteristics. First data corresponding to the first region and second data corresponding to the second region are extracted from the deep layer point cloud data, and third data corresponding to the first region and fourth data corresponding to the second region are extracted from the local fish swarm 3D point cloud data. Based on the data in the local fish swarm 3D point cloud data, the first data and the third data are fused, and the second data and the fourth data are weighted and fused. That is, when fusing the first data in the depth layer point cloud data and the third data in the local fish swarm 3D point cloud data, since the third data has better accuracy than the first data while also ensuring data comprehensiveness, in order to improve the fusion processing efficiency, this embodiment preferably uses the third data directly as the point cloud data of the fused first region; when fusing the second data in the depth layer point cloud data and the fourth data in the local fish swarm 3D point cloud data, the third data is weighted and fused. While the second data has a wider coverage (comprehensive data) but slightly lower accuracy, and the fourth data has higher accuracy but relatively less comprehensive data, this embodiment preferably acquires the point cloud density of the second region from the depth layer point cloud data and the local fish swarm 3D point cloud data, respectively, in order to fully utilize the useful information of the second and fourth data. Then, it sets corresponding fusion weights (the ratio of the point cloud density to the sum of the point cloud densities of the two types of data) based on the point cloud densities of the second and fourth data, and uses the obtained fusion weights to perform weighted fusion of the second and fourth data to obtain the fused point cloud data of the second region. The point cloud data obtained by this fusion method, which includes the data information of both the fused first and second regions, is the point cloud fusion result (local fused fish swarm point cloud data) obtained after executing each fusion scheme.

[0050] The point cloud fusion results after executing each fusion scheme are integrated to obtain the 3D point cloud data of the target fish school. In practical applications, the point cloud fusion results obtained after executing each fusion scheme are sequentially stitched together according to the corresponding point cloud depth layers to obtain the required 3D point cloud data of the target fish school, which will not be described in detail here.

[0051] This embodiment not only effectively ensures the non-interference and high efficiency of point cloud data fusion at different depths by using a fusion mechanism that combines local point cloud data with global point cloud layering and local point cloud fusion, but also effectively ensures the overall coverage integrity, high quality and high accuracy of the fused point cloud data by using local point cloud data as the standard in dense areas and point cloud density-based fusion weights in non-dense areas.

[0052] S15. Perform 3D reconstruction based on the target fish group's 3D point cloud data to obtain the corresponding 3D model of the fish group, and then visualize the 3D model of the fish group.

[0053] Specifically, the steps of performing 3D reconstruction based on the target fish school's 3D point cloud data to obtain a corresponding 3D model of the fish school, and then visualizing the 3D model of the fish school, include: The three-dimensional point cloud data of the target fish group is preprocessed to obtain the three-dimensional point cloud data to be analyzed. The preprocessing includes background filtering, outlier removal and normalization. The specific processing procedure is implemented with reference to relevant existing technologies and will not be described in detail here.

[0054] Based on the 3D point cloud data to be analyzed, corresponding individual fish point clouds are obtained using a preset point cloud segmentation deep learning model. The preset point cloud segmentation deep learning model can be understood as a model capable of segmenting point cloud instances, such as PointNet++ or PointCNN, to address the problem of overlapping / adhesion among fish. It should be noted that the training and construction of the preset point cloud segmentation deep learning model can refer to existing technologies. After obtaining multiple point cloud instances based on the preset point cloud segmentation deep learning model, considering that fish bodies may occlude in actual fish schools, some point cloud instances may have missing data holes and cannot be used for subsequent analysis. Preferably, in this embodiment, after obtaining multiple point cloud instances based on the preset point cloud segmentation deep learning model, hole-filling processing is performed on each point cloud instance (for example, using moving least squares to fill missing values) to obtain individual fish point clouds that are convenient for subsequent analysis.

[0055] The current static features of each individual fish point cloud are obtained. These static features include fish position, geometric features, posture features, and point cloud descriptors. Fish position refers to the location of the individual fish point cloud within the target fish group's 3D point cloud data. Geometric features include centroid, body length, body width, and volume, which can be calculated using point cloud convex hull or principal component analysis (PCA). Specific calculation processes can refer to existing technologies. Posture features include orientation and curvature. Orientation can be understood as the principal axis direction determined by PCA, and curvature can be defined using the curvature of the fitted curve of the point cloud data, or based on the ratio of the principal axis length to the secondary axis length determined by PCA; no specific limitations are made here. Point cloud descriptors include a Fast Point Feature Histogram (FPFH) and global features (point cloud distribution entropy, symmetry, etc.). Specific acquisition methods can also refer to existing technologies, such as using point cloud libraries like PCL or Open3D for calculation; details are omitted here.

[0056] The historical static features of the fish body at the previous few sampling times corresponding to each individual fish point cloud are combined with the current static features of the fish body in a temporal sequence to obtain the temporal features of the corresponding individual fish point cloud. The historical static features of the fish body at the previous few sampling times can be obtained by associating the same individual fish body with existing target tracking methods based on point cloud data. The specific acquisition process can be referred to relevant existing technologies.

[0057] Based on the temporal features of individual fish point clouds, behavior recognition is performed using a pre-built behavior classification model to obtain the corresponding behavior category. The type of behavior classification model can be selected based on actual application requirements. For example, models that effectively analyze temporal features, such as Long Short-Term Memory (LSTM) models and Transformer models, can be used. These models can be pre-trained using temporal data of static fish features collected under different behavior categories (including "cruising," "escape," and "foraging"). Specific training methods vary depending on the model type and will not be detailed here. It should be noted that before inputting the temporal features of individual fish point clouds into the behavior classification model, appropriate preprocessing (such as normalization and feature vector encoding) is required, depending on the model type, before inputting into the behavior classification model for behavior recognition and analysis.

[0058] Based on the Poisson reconstruction algorithm, the surface of each individual fish point cloud is reconstructed, and the fish school 3D model is obtained based on the obtained individual fish 3D model and the corresponding fish body position. The reconstruction process of the individual fish 3D model can be implemented with reference to existing Poisson reconstruction technology, which will not be described in detail here. When combining the corresponding individual fish 3D models based on the fish body positions to obtain the fish school 3D model, attention should be paid to coordinate alignment to ensure the accuracy of the fish school structure.

[0059] Based on the current static features and behavioral categories of each individual fish point cloud, the 3D model of the fish school is rendered and displayed. During actual rendering, coloring can be used to distinguish behaviors, such as red representing "foraging," green representing "cruising," and yellow representing "escaping." Geometric and posture features in the current static features of the fish can be labeled, and the movement trajectory of each fish (connecting the centroids at different sampling times), the total number of fish, and the average fish size can also be displayed. Actual rendering can be implemented using existing visualization libraries and engines, such as Unity3D and OpenGL, to achieve a visual and intuitive display of the 3D model of the fish school and its corresponding ecological characteristics. This allows fish farmers to understand the activity status, ecological characteristics, and health status of the fish in the cages in real time, providing a reliable basis for timely risk warnings, feeding management optimization, and the formulation of fishing plans.

[0060] This invention provides a method for deploying a multibeam sonar at the top geometric center of a target fish cage and multiple multibeam sonar probes on the sides of the target fish cage at multiple key depth layers. Based on the multibeam sonar, global 3D point cloud data is acquired. After acquiring corresponding local 2D point cloud data from each multibeam sonar probe, the global 3D point cloud data is converted into corresponding global fish swarm 3D point cloud data based on a preset fish cage topology coordinate system. Each set of local 2D point cloud data is then converted into corresponding local fish swarm 3D point cloud data. Based on the content information between the global fish swarm 3D point cloud data and each local fish swarm 3D point cloud data, various fusion schemes are determined and executed to obtain the target fish swarm 3D point cloud data. Finally, 3D reconstruction is performed based on the target fish swarm 3D point cloud data to obtain the corresponding fish swarm 3D model. The scheme for visualizing a 3D model of a fish school utilizes a three-dimensional monitoring architecture. This architecture employs a multi-beam sonar array deployed at the geometric center of the top of the fish cage to provide a global 3D overview of the entire water space. It also utilizes multiple multi-beam sonar probes deployed at key depths along the cage's sidewalls to provide a high-resolution, close-range 2D scanning perspective at specific depths. This combined global overhead view with a high-precision local side-view perspective, along with a reliable fusion mechanism of global 3D point clouds and local high-resolution 2D point clouds, ensures not only the integrity of the overall spatial detection of the fish school but also the refinement of local detail detection. This effectively improves the reconstruction quality and visualization effect of the 3D model of the fish school, providing a reliable analytical basis for accurately assessing the fish population, size distribution, behavior, health status, and cage safety.

[0061] It should be noted that although the steps in the flowchart above are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order requirement for the execution of these steps, and they can be executed in other orders.

[0062] In one embodiment, such as Figure 2 As shown, a sonar-based 3D modeling system for deep-sea cage aquaculture fish schools is provided. A multibeam sonar array is deployed at the geometric center of the top of the target cage, and multiple multibeam sonar probes are deployed on the sides of the target cage at multiple key depth layers. The system includes: The data acquisition module 1 is used to acquire global three-dimensional point cloud data based on the multibeam sonar, and acquire corresponding local two-dimensional point cloud data based on each of the multibeam sonar probes. Point cloud conversion module 2 is used to convert the global three-dimensional point cloud data into corresponding global fish swarm three-dimensional point cloud data based on a preset cage topology coordinate system, and to convert each group of local two-dimensional point cloud data into corresponding local fish swarm three-dimensional point cloud data. The fusion scheme generation module 3 is used to determine each fusion scheme based on the content information between the global fish swarm 3D point cloud data and each of the local fish swarm 3D point cloud data. The point cloud fusion module 4 is used to execute each of the fusion schemes to obtain the three-dimensional point cloud data of the target fish group; wherein, each of the fusion schemes includes single-parameter controlled fusion for a first region and dual-parameter collaborative fusion for a second region, the first region is determined based on the content information, and the second region is a region other than the first region; The 3D reconstruction module 5 is used to perform 3D reconstruction based on the 3D point cloud data of the target fish group to obtain the corresponding 3D model of the fish group, and to visualize the 3D model of the fish group.

[0063] Specific limitations regarding the sonar-based 3D modeling system for deep-sea fish cage culture can be found in the above section on the limitations of the sonar-based 3D modeling method for deep-sea fish cage culture; the corresponding technical effects are equivalent and will not be repeated here. Each module in the aforementioned sonar-based 3D modeling system for deep-sea fish cage culture can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the computer device's memory as software, allowing the processor to call and execute the corresponding operations of each module.

[0064] In summary, the present invention provides a sonar-based three-dimensional modeling method and system for deep-sea cage aquaculture fish schools. The sonar-based three-dimensional modeling method utilizes a multi-beam sonar array deployed at the geometric center of the cage top to provide a global three-dimensional overview of the entire cage's water space. It also employs a three-dimensional cross-monitoring architecture that combines a global overhead view with a local high-precision side-view by deploying multiple multi-beam sonar probes at key depth layers on the cage's sidewalls. This, combined with a reliable fusion mechanism of global three-dimensional point clouds and local high-resolution two-dimensional point clouds, not only ensures the integrity of the global spatial detection of the cage's water body but also guarantees the refinement of local detail detection. This effectively improves the reconstruction quality and visualization effect of the three-dimensional fish school model, providing a reliable analytical basis for accurately assessing the fish's quantity, size distribution, behavior, health status, and cage safety.

[0065] The various embodiments in this specification are described in a progressive manner. For directly identical or similar parts of the embodiments, refer to each other. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. It should be noted that the technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0066] The above-described embodiments are merely preferred embodiments of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various improvements and substitutions without departing from the principles of the present invention, and these improvements and substitutions should also be considered within the scope of protection of the present invention. Therefore, the scope of protection of this invention should be determined by the scope of the claims.

Claims

1. A method for three-dimensional modeling of fish schools in deep-sea cage culture based on sonar, characterized in that, A multi-beam sonar array is deployed at the top geometric center of the target cage, and multiple multi-beam sonar probes are deployed on the sides of multiple key depth layers of the target cage. The method includes: Global three-dimensional point cloud data is acquired based on the multibeam sonar, and corresponding local two-dimensional point cloud data is acquired based on each of the multibeam sonar probes; the local two-dimensional point cloud data includes the horizontal plane coordinates and echo intensity of each detection point in different key depth layers; the horizontal plane coordinates do not have Cartesian coordinates with a Z-axis. Based on the preset cage topology coordinate system, the global three-dimensional point cloud data is converted into the corresponding global fish swarm three-dimensional point cloud data, and each group of local two-dimensional point cloud data is converted into the corresponding local fish swarm three-dimensional point cloud data. Based on the content information between the global fish swarm 3D point cloud data and each of the local fish swarm 3D point cloud data, various fusion schemes are determined; the content information includes at least the effective depth coverage range; determining each fusion scheme includes: Each point cloud depth layer is obtained based on the installation depth of each multibeam sonar probe and the effective coverage range of the depth. Based on each point cloud depth layer, the global fish swarm 3D point cloud data is divided into point cloud data of each depth layer; Determine the fusion scheme for fusing the deep layer point cloud data with the corresponding local fish swarm 3D point cloud data; Each of the aforementioned fusion schemes is executed to obtain three-dimensional point cloud data of the target fish swarm; wherein, each of the aforementioned fusion schemes includes single-parameter controlled fusion for a first region and dual-parameter collaborative fusion for a second region, the first region being determined based on the content information, and the second region being a region other than the first region; A 3D reconstruction is performed based on the 3D point cloud data of the target fish school to obtain a corresponding 3D model of the fish school, and the 3D model of the fish school is then visualized.

2. The sonar-based three-dimensional modeling method for deep-sea cage-cultured fish schools as described in claim 1, characterized in that, The global three-dimensional point cloud data includes the Cartesian coordinates and echo intensity of different detection points; The steps for acquiring global three-dimensional point cloud data based on the multibeam sonar include: According to the preset detection frequency, the first raw beam data is collected by the multi-beam sonar. Based on the marine environmental monitoring data at the sampling time corresponding to the first original beam data, the actual sound ray path is obtained by performing sound ray bending correction based on the ray tracing method; the marine environmental monitoring data is obtained based on environmental sensor arrays deployed at various key depth layers; the environmental sensor arrays include temperature sensors, salinity sensors and pressure sensors. Based on the actual sound path, the slant range, horizontal angle, and pitch angle in the first original beam data are corrected to obtain the first corrected beam data. The first corrected beam data is transformed into Cartesian coordinates to obtain the global three-dimensional point cloud data.

3. The sonar-based three-dimensional modeling method for deep-sea cage aquaculture fish schools as described in claim 2, characterized in that, The marine environmental monitoring data includes temperature, salinity, and depth at various key depth layers; The step of obtaining the actual sound ray path by performing sound ray bending correction based on the marine environmental monitoring data corresponding to the sampling time of the first original beam data includes: Based on the temperature, salinity, and depth of each key depth layer, the actual sound speed of the corresponding key depth layer is calculated using a preset classical ocean sound speed model. Based on the actual sound velocity of each of the key depth layers, a sound velocity profile is constructed using a preset interpolation algorithm. Based on the sound velocity profile, the actual sound ray path is obtained by performing sound ray bending correction using the ray tracing method.

4. The sonar-based three-dimensional modeling method for deep-sea cage-cultured fish schools as described in claim 1, characterized in that, The global three-dimensional point cloud data includes the Cartesian coordinates and echo intensity of different detection points; The step of converting the global 3D point cloud data into corresponding global fish swarm 3D point cloud data based on a preset cage topology coordinate system includes: The pre-calibrated sonar installation deflection angle is summed with the component data of the corresponding acquired cage attitude angle to calculate the comprehensive rotation angle; the sonar installation deflection angle includes lateral deflection angle and longitudinal deflection angle. Based on the comprehensive rotation angle, a first rotation matrix is ​​constructed, and attitude compensation is performed on the global three-dimensional point cloud data according to the first rotation matrix to obtain attitude-compensated three-dimensional point cloud data. Based on the rotation angle of the main symmetry axis of the cage, the attitude compensation three-dimensional point cloud data is corrected by the cage rotation angle to obtain rotation angle corrected three-dimensional point cloud data; the rotation angle of the main symmetry axis of the cage is calculated based on multiple acoustic beacons deployed on the rigid structure of the target cage; Based on a preset echo intensity range, background noise is removed from the rotation angle-corrected 3D point cloud data to obtain the global fish swarm 3D point cloud data.

5. The sonar-based three-dimensional modeling method for deep-sea cage-cultured fish schools as described in claim 1, characterized in that, Based on a preset cage topology coordinate system, the steps for converting the local two-dimensional point cloud data of each group into corresponding local three-dimensional point cloud data of the fish school include: Based on the installation depth and beam vertical opening angle of each of the multibeam sonar probes, the corresponding local two-dimensional point cloud data is stretched in the depth direction to obtain the corresponding pseudo local three-dimensional point cloud data. Based on the calibration position and probe attitude angle of each of the multibeam sonar probes in the preset cage topology coordinate system, the corresponding coordinate transformation matrix is ​​obtained; Based on the coordinate transformation matrix corresponding to the multibeam sonar probe, the corresponding pseudo-local 3D point cloud data is transformed to obtain the corresponding candidate local 3D point cloud data. Based on a preset echo intensity range, background noise is removed from the candidate local 3D point cloud data to obtain the local fish swarm 3D point cloud data.

6. The sonar-based three-dimensional modeling method for deep-sea cage-cultured fish schools as described in claim 5, characterized in that, The step of stretching the corresponding local two-dimensional point cloud data in the depth direction based on the installation depth and beam vertical opening angle of each of the multi-beam sonar probes to obtain the corresponding pseudo-local three-dimensional point cloud data includes: Based on the vertical beam opening angle and installation depth of each of the multibeam sonar probes, the corresponding total coverage range in the vertical direction of the beam is obtained based on the geometric relationship between the beam centerline and the coverage radius. Based on the total vertical coverage of each multibeam sonar probe and the installation depth, the corresponding effective depth coverage is obtained. Based on the effective depth coverage range and preset depth change step size of each of the multibeam sonar probes, the depth direction coordinates of each two-dimensional plane coordinate in the corresponding local two-dimensional point cloud data are serialized and expanded to generate the corresponding pseudo local three-dimensional point cloud data.

7. The sonar-based three-dimensional modeling method for deep-sea cage aquaculture fish schools as described in claim 1, characterized in that, The content information also includes the morphological characteristics of the fish school; The execution of each of the aforementioned fusion schemes yields 3D point cloud data of the target fish swarm, including: During the execution of each fusion scheme, the first region and the second region are determined based on the fish morphological characteristics. First data corresponding to the first region and second data corresponding to the second region are extracted from the deep layer point cloud data, and third data corresponding to the first region and fourth data corresponding to the second region are extracted from the local fish swarm 3D point cloud data. Based on the data in the local fish swarm 3D point cloud data, the first data and the third data are fused, and the second data and the fourth data are weighted and fused. By integrating the point cloud fusion results after each of the aforementioned fusion schemes is executed, the three-dimensional point cloud data of the target fish group is obtained.

8. The sonar-based three-dimensional modeling method for deep-sea cage aquaculture fish schools as described in claim 1, characterized in that, The steps of performing 3D reconstruction based on the target fish school's 3D point cloud data to obtain a corresponding 3D model of the fish school, and then visualizing the 3D model of the fish school, include: The target fish group's 3D point cloud data is preprocessed to obtain the 3D point cloud data to be analyzed; the preprocessing includes background filtering, outlier removal, and normalization. Based on the three-dimensional point cloud data to be analyzed, the corresponding fish individual point cloud is obtained based on a preset point cloud segmentation deep learning model. Based on the point cloud of each individual fish, the corresponding current static features of the fish body are obtained; the current static features of the fish body include the fish body position, geometric features, posture features, and point cloud descriptors; The historical static features of the fish body at the previous few sampling times corresponding to each individual fish point cloud are combined with the current static features of the fish body in a temporal sequence to obtain the corresponding temporal features of the individual fish point cloud. The point cloud descriptor time series data and fish body posture feature time series data in the fish individual point cloud time series features corresponding to each fish individual point cloud are input into the pre-built behavior classification model for behavior recognition to obtain the corresponding behavior category. Based on the Poisson reconstruction algorithm, surface reconstruction is performed on the point cloud of each individual fish to obtain a three-dimensional model of a single fish. Based on the three-dimensional model of the single fish and the corresponding position of the fish body, the three-dimensional model of the fish group is obtained. Based on the current static features and behavioral categories of each individual fish point cloud, the 3D model of the fish school is rendered and displayed.

9. A three-dimensional modeling system for deep-sea cage-cultured fish schools based on sonar, characterized in that, A multibeam sonar array is deployed at the top geometric center of the target cage, and multiple multibeam sonar probes are deployed on the sides of the target cage at multiple key depth layers. The system includes: The data acquisition module is used to acquire global three-dimensional point cloud data based on the multibeam sonar, and to acquire corresponding local two-dimensional point cloud data based on each of the multibeam sonar probes; the local two-dimensional point cloud data includes the horizontal plane coordinates and echo intensity of each detection point in different key depth layers; the horizontal plane coordinates do not have Cartesian coordinates with a Z-axis. The point cloud conversion module is used to convert the global three-dimensional point cloud data into the corresponding global fish swarm three-dimensional point cloud data based on the preset cage topology coordinate system, and to convert each group of local two-dimensional point cloud data into the corresponding local fish swarm three-dimensional point cloud data. A fusion scheme generation module is used to determine various fusion schemes based on the content information between the global fish swarm 3D point cloud data and each of the local fish swarm 3D point cloud data; the content information includes at least the effective depth coverage range; determining various fusion schemes includes: Each point cloud depth layer is obtained based on the installation depth of each multibeam sonar probe and the effective coverage range of the depth. Based on each point cloud depth layer, the global fish swarm 3D point cloud data is divided into point cloud data of each depth layer; Determine the fusion scheme for fusing the deep layer point cloud data with the corresponding local fish swarm 3D point cloud data; The point cloud fusion module is used to execute each of the fusion schemes to obtain the three-dimensional point cloud data of the target fish swarm; wherein, each of the fusion schemes includes single-parameter controlled fusion for a first region and dual-parameter collaborative fusion for a second region, the first region being determined based on the content information, and the second region being a region other than the first region; The 3D reconstruction module is used to perform 3D reconstruction based on the 3D point cloud data of the target fish group to obtain the corresponding 3D model of the fish group, and to visualize the 3D model of the fish group.

Images