A method, device and system for three-dimensional modeling of water body based on underwater robot

Abstract

The present invention discloses a method, device and system for three-dimensional modeling of a water body based on an underwater robot, and relates to the field of computer technology. The method comprises: the underwater robot uses an inertial navigation system, a dynamic obstacle avoidance algorithm and a GPS positioning technology to move to a preset operation target point; obtains underwater and surface images and water body parameters, and transmits them to a cloud platform; performs nonlinear filtering processing on the underwater image and the surface image, and performs noise reduction processing on the water body parameters; determines aquatic targets based on the water body parameters and the underwater image, and performs dynamic tracking and refinement scanning on the aquatic targets; calculates the optical flow of the surface image based on the surface image using a motion estimation neural network; constructs a three-dimensional model of the water environment based on the underwater image, the water body parameters and the dynamic tracking and refinement scanning results, and establishes a decision model based on the three-dimensional model of the water environment and the optical flow of the surface image. The present invention enables the three-dimensional modeling of the water environment to achieve a more ideal effect, thereby improving the benefits of aquaculture.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method, device and system for three-dimensional modeling of a water body based on an underwater robot. Background Art

[0002] Underwater robots are a kind of flexible underwater operation equipment. Compared with traditional underwater operation means, they have obvious advantages in operation mode, scope and adaptability. Using underwater robots for operation can improve operation efficiency and quality. Three-dimensional modeling of water bodies is an extremely important part of underwater operations. However, in the existing three-dimensional modeling methods of water bodies based on underwater robots, when the underwater robot moves, the position parameters are usually only calculated by the inertial navigation system. However, due to the inevitable system drift generated over time, the calculated position parameters will continue to accumulate over time, resulting in a large error in the obtained position parameters, making it impossible for the underwater robot to move accurately along the path. For the three-dimensional modeling of water bodies, accurate identification of aquatic targets is very important, but currently only a single robot vision acquisition or a single laser scanning is usually used, resulting in the difficulty of identifying aquatic targets. At the same time, the filtering processing of the acquired underwater images and the noise reduction processing of water parameters are also technical difficulties that need to be overcome. If the image and parameter processing are not complete, it will cause a large error in the final identification of aquatic targets, resulting in the failure of the three-dimensional modeling of the water environment to achieve the ideal effect. Summary of the invention

[0003] The purpose of the present invention is to overcome the deficiencies of the prior art. The present invention provides a method, device and system for three-dimensional modeling of water bodies based on an underwater robot, so that the three-dimensional modeling of the water environment can achieve a more ideal effect, thereby improving the efficiency of aquaculture.

[0004] In order to solve the above technical problems, the present invention provides a method for three-dimensional modeling of a water body based on an underwater robot, the method comprising:

[0005] Starting the underwater robot, which moves to each preset operation target point based on a preset operation path using an inertial navigation system, a dynamic obstacle avoidance algorithm, and GPS positioning technology;

[0006] The underwater robot uses a video acquisition system to obtain underwater images and surface images at the preset operation target point, uses a plurality of sensors to obtain water body parameters, and transmits the underwater images, surface images and water body parameters to a cloud platform;

[0007] The cloud platform performs nonlinear filtering on the underwater image and the surface image to obtain the underwater image and the surface image after the nonlinear filtering, and performs noise reduction on the water body parameters to obtain the water body parameters after the noise reduction;

[0008] The cloud platform determines the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and controls the underwater robot to perform dynamic tracking and refinement scanning on the aquatic target to obtain dynamic tracking and refinement scanning results;

[0009] The cloud platform calculates the optical flow of the water surface image using a motion estimation neural network based on the water surface image after nonlinear filtering;

[0010] A three-dimensional model of the water environment is constructed based on the underwater image after nonlinear filtering, the water parameters after noise reduction and the dynamic tracking and refinement scanning results, and a decision model is established based on the three-dimensional model of the water environment and the optical flow of the water surface image.

[0011] Optionally, the underwater robot uses an inertial navigation system, a dynamic obstacle avoidance algorithm and GPS positioning technology to move to each preset operation target point, including:

[0012] When the underwater robot moves based on a preset operation path, initial current position parameters of the underwater robot are calculated based on the inertial navigation system;

[0013] Performing error correction on the initial current position parameter based on GPS positioning technology to obtain a corrected position parameter;

[0014] Based on the corrected position parameters, a dynamic obstacle avoidance algorithm is used to perform local path planning to obtain a local operating path, and the preset operating path is adjusted based on the local operating path.

[0015] Optionally, transmitting the underwater image, the surface image and the water body parameters to a cloud platform includes:

[0016] The water body parameters, water surface images and underwater images are transmitted to a cloud platform using a fuzzy control algorithm based on a wireless sensor network.

[0017] Optionally, performing nonlinear filtering on the underwater image and the surface image to obtain the underwater image and the surface image after the nonlinear filtering includes:

[0018] Performing grayscale processing on the underwater image and the water surface image to obtain a grayscale underwater image and a grayscale water surface image;

[0019] Calculating a first grayscale value and a first grayscale mean of the grayscale underwater image, setting a first grayscale threshold based on the first grayscale mean, and performing weighted mean filtering on the grayscale underwater image using a weight coefficient of a nonlinear distribution based on the first grayscale value and the first grayscale threshold to obtain an underwater image after nonlinear filtering;

[0020] Calculate the second grayscale value and the second grayscale mean of the grayscale water surface image, set the second grayscale threshold based on the second grayscale mean, perform weighted mean filtering on the grayscale water surface image using nonlinearly distributed weight coefficients based on the second grayscale value and the second grayscale threshold, and obtain the water surface image after nonlinear filtering.

[0021] Optionally, performing noise reduction processing on the water body parameters to obtain the water body parameters after noise reduction processing includes:

[0022] Performing wavelet decomposition on the water body parameters to obtain corresponding high-frequency wavelet coefficients and low-frequency wavelet coefficients;

[0023] Calculating the noise variance based on the high-frequency wavelet coefficients and the low-frequency wavelet coefficients, and scaling the high-frequency wavelet coefficients and the low-frequency wavelet coefficients based on the noise variance to obtain scale-transformed high-frequency wavelet coefficients and low-frequency wavelet coefficients;

[0024] Based on the high-frequency wavelet coefficients and low-frequency wavelet coefficients after scale transformation, wavelet inverse transform is performed to obtain the water body parameters after noise reduction processing.

[0025] Optionally, the cloud platform determines the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and controls the underwater robot to perform dynamic tracking and refinement scanning on the aquatic target to obtain dynamic tracking and refinement scanning results, including:

[0026] Extracting a micro-Doppler spectrum of the water body parameters after noise reduction processing, and performing feature extraction based on the micro-Doppler spectrum to obtain a micro-Doppler feature;

[0027] Perform feature extraction on underwater images after nonlinear filtering to obtain image semantic features;

[0028] The micro-Doppler feature and the image semantic feature are subjected to feature fusion to obtain a target fusion feature, and target recognition is performed based on the target fusion feature to determine the aquatic target;

[0029] After determining the aquatic target, the cloud platform adjusts the moving path and scanning parameters of the underwater robot. The underwater robot performs dynamic tracking and detailed scanning on the aquatic target based on the adjusted moving path and scanning parameters in combination with the LightCrafter4500 optical projector to obtain dynamic tracking and detailed scanning results.

[0030] Optionally, the cloud platform calculates the optical flow of the water surface image using a motion estimation neural network based on the water surface image after nonlinear filtering, including:

[0031] Perform feature extraction on the water surface image after nonlinear filtering to obtain a feature map;

[0032] The feature map is input into a motion estimation neural network, and the optical flow of the water surface image is calculated based on the motion estimation neural network, wherein the motion estimation neural network is a convergence model obtained by inputting a sample data set into a deep neural network for training, and the deep neural network includes a channel attention mechanism module, a spatial attention module, an iterative update module and an upsampling module.

[0033] Optionally, the three-dimensional model of the water environment is constructed based on the underwater image after nonlinear filtering, the water body parameters after noise reduction processing and the dynamic tracking and refinement scanning results, and the decision model is established based on the underwater three-dimensional model and the optical flow of the water surface image, including:

[0034] Extracting modeling parameters based on underwater images after nonlinear filtering, water body parameters after noise reduction, and dynamic tracking and refinement scanning results, and constructing a three-dimensional model of the water environment using digital twin technology based on the modeling parameters;

[0035] Based on the three-dimensional model of the water environment combined with the digital twin, a visual simulation is performed to obtain a visual simulation result, and global data information is obtained based on the visual simulation result and the optical flow of the water surface image, and a decision model is established based on the global data information.

[0036] In addition, the present invention also provides a water body three-dimensional modeling device based on an underwater robot, the device comprising:

[0037] Underwater robot moving module: used to start the underwater robot, which moves to each preset operation target point based on the preset operation path using an inertial navigation system, a dynamic obstacle avoidance algorithm and GPS positioning technology;

[0038] Image and parameter acquisition module: used for the underwater robot to obtain underwater images and surface images at the preset operation target point using a video acquisition system, obtain water body parameters using a number of sensors, and transmit the underwater images, surface images and water body parameters to the cloud platform;

[0039] Image and parameter processing module: used for the cloud platform to perform nonlinear filtering on the underwater image and the surface image to obtain the underwater image and the surface image after the nonlinear filtering, and to perform noise reduction on the water body parameters to obtain the water body parameters after the noise reduction;

[0040] Tracking and refining scanning module: used for the cloud platform to determine the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and control the underwater robot to perform dynamic tracking and refining scanning on the aquatic target to obtain dynamic tracking and refining scanning results;

[0041] Water surface image optical flow module: used for the cloud platform to calculate the water surface image optical flow using a motion estimation neural network based on the water surface image after nonlinear filtering;

[0042] 3D modeling module: used to build a 3D model of the water environment based on the underwater image after nonlinear filtering, the water parameters after noise reduction and the dynamic tracking and refinement scanning results, and to establish a decision model based on the 3D model of the water environment and the optical flow of the water surface image.

[0043] In addition, the present invention also provides a three-dimensional water body modeling system based on an underwater robot, the system includes an underwater robot and a cloud platform, the underwater robot includes a control mainboard, a power supply, a sensor, a camera, a wireless communication system and a motion control system, wherein the control mainboard is connected to the power supply, the sensor, the camera, the wireless communication system and the motion control system, respectively, the cloud platform establishes a communication connection with the wireless communication system of the underwater robot, and the system is configured to execute the above-mentioned three-dimensional water body modeling method based on the underwater robot.

[0044] In an embodiment of the present invention, the error of the inertial navigation system is continuously corrected through a dynamic obstacle avoidance algorithm and GPS positioning technology to continuously obtain accurate position parameters, so that the local path can be adjusted in time to avoid deviation from the operation target point, and nonlinear filtering processing of the image is performed using nonlinearly distributed weight coefficients to make the obtained image clearer. At the same time, wavelet transform is used to perform noise reduction processing on water parameters to make the obtained water parameters more accurate. The optical flow of the water surface image is calculated through a motion estimation network to reflect the feeding intensity of fish. At the same time, digital twin technology is used to establish a three-dimensional model of the water environment, so that the established three-dimensional model of the water environment can more accurately reflect the actual state of the aquaculture area. Combining the three-dimensional model of the water environment and the optical flow of the water surface image can establish a decision-making model to achieve scientific and intelligent decision-making in aquatic areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0046] Figure 1 is a schematic flow chart of a method for three-dimensional modeling of a water body based on an underwater robot in an embodiment of the present invention;

[0047] Figure 2 Schematic diagram of the structure of a three-dimensional water modeling device based on an underwater robot in an embodiment of the present invention;

[0048] Figure 3 It is a schematic diagram of the structural composition of a water body three-dimensional modeling system based on an underwater robot in an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] Embodiment 1

[0051] See also Figure 1 , Figure 1 It is a flow chart of a method for three-dimensional modeling of a water body based on an underwater robot in an embodiment of the present invention.

[0052] like Figure 1 As shown, a method for three-dimensional modeling of a water body based on an underwater robot, the method comprising:

[0053] S11: Starting the underwater robot, which moves to each preset operation target point based on a preset operation path using an inertial navigation system, a dynamic obstacle avoidance algorithm, and GPS positioning technology;

[0054] In the specific implementation process of the present invention, the underwater robot uses an inertial navigation system, a dynamic obstacle avoidance algorithm and GPS positioning technology to move to each preset operation target point, including: when the underwater robot moves based on a preset operation path, the initial current position parameters of the underwater robot are calculated based on the inertial navigation system; the initial current position parameters are error corrected based on the GPS positioning technology to obtain corrected position parameters; local path planning is performed based on the corrected position parameters using a dynamic obstacle avoidance algorithm to obtain a local operation path, and the preset operation path is adjusted based on the local operation path.

[0055] Specifically, the user can start the underwater robot through the terminal device and set the operation path of the underwater robot. When the underwater robot moves based on the preset operation path, the inertial navigation system belongs to the dead reckoning navigation mode, that is, the position of the next point is calculated from the position of a known point according to the continuously measured heading angle and speed of the moving body, so the initial current position parameters of the underwater robot can be continuously measured. However, since the navigation information is generated through integration, its positioning error increases with time; therefore, based on the Global Positioning System (GPS), the positioning error of the underwater robot is gradually increased. System, GPS) positioning technology is used to perform error correction on the initial current position parameters. The GPS positioning technology can provide an accurate geographic spatial position through a high-precision positioning system. The position parameters calculated by the inertial navigation system are corrected by the GPS positioning technology to obtain the corrected position parameters, which are the accurate actual position parameters of the underwater robot. Although the position parameters have been corrected and the accurate actual position parameters of the underwater robot are known, the actual position of the underwater robot has shifted. Based on the corrected position parameters, a dynamic obstacle avoidance algorithm is used to perform local path planning, with the current position point as the starting point and the corresponding next point in the preset operation path as the target point. A plurality of path nodes are set according to the starting point and the target point, and the cost value of the target point is set to zero, and the cost value of all path nodes is set to zero. The cost value is set to infinity, starting from the target point, searching for a path to the starting point according to the heuristic rule, updating the path cost value according to the cost value of the neighboring nodes of the current path node, tracing back from the target point to the starting point along the updated path, updating the cost value of the neighboring nodes of each path node on the path, in each update, detecting whether the change of the cost value of the path node exceeds the preset threshold, if it exceeds the preset threshold, searching for another path node, repeating the above process until the preset number of iterations is reached, and obtaining the local operation path, adjusting the preset operation path based on the local operation path, replacing the corresponding part of the operation path with the local operation path, thereby adjusting the preset operation path so that the underwater robot can continue to move along each preset operation target point.

[0056] S12: The underwater robot obtains underwater images and surface images at the preset operation target point using a video acquisition system, obtains water body parameters using a plurality of sensors, and transmits the underwater images, surface images and water body parameters to a cloud platform;

[0057] In the specific implementation process of the present invention, the transmitting of the underwater image, the surface image and the water body parameters to the cloud platform includes: transmitting the water body parameters, the surface image and the underwater image to the cloud platform using a fuzzy control algorithm based on a wireless sensor network.

[0058] Specifically, the underwater robot uses a video acquisition system to obtain underwater images and surface images at each preset operation target point. The video acquisition system uses a charge coupled device (CDD) industrial camera and a LightCrafter 4500 optical projector, and uses a number of sensors to obtain water parameters. The water parameters include signal parameters and temperature parameters, etc. The several sensors include dissolved oxygen sensors, ammonia nitrogen sensors, ultrasonic ranging sensors, pH sensors, temperature sensors, turbidity sensors, and gravity sensors, etc. Based on a wireless sensor network, a fuzzy control algorithm is used to transmit the water parameters, surface images, and underwater images to a cloud platform, and the water parameters, surface images, and underwater images are compressed and packaged to obtain packet data, and a delay rate of packet data transmission is calculated. The delay rate is fuzzy controlled by a fuzzy control algorithm, that is, the delay rate is fuzzified to obtain a set of precise quantities of transmission delay and a set of fuzzy quantities corresponding to the precise quantities, and the calculation is performed. The membership between the set of precise quantities of transmission delay and the set of fuzzy quantities corresponding to the precise quantities is obtained according to the membership, and the fuzzy subset is obtained according to the membership. The fuzzy control output is inferred by the fuzzy subset using fuzzy control logic, and the fuzzy control output is defuzzified to obtain the corresponding fuzzy control result. The fuzzy control result includes a suitable delay and a transfer function. The fuzzy control algorithm is used to solve the problem of data lag, so that data transmission is faster and smoother; according to the fuzzy control result, the packet data is transmitted to the cloud platform by using a wireless sensor network. The wireless sensor network is a wireless network composed of a large number of sensors in a self-organizing and multi-hop manner, which senses, collects, processes and transmits information of sensed objects in the geographical area covered by the network in a collaborative manner.

[0059] S13: the cloud platform performs nonlinear filtering on the underwater image and the surface image to obtain the underwater image and the surface image after the nonlinear filtering, and performs noise reduction on the water body parameters to obtain the water body parameters after the noise reduction;

[0060] In the specific implementation process of the present invention, the underwater image and the surface image are subjected to nonlinear filtering processing to obtain the underwater image and the surface image after the nonlinear filtering processing, including: graying the underwater image and the surface image to obtain the grayed underwater image and the grayed surface image; calculating the first gray value and the first gray mean of the grayed underwater image, and setting the first gray threshold based on the first gray mean, and performing weighted mean filtering of the grayed underwater image using the weight coefficient of nonlinear distribution based on the first gray value and the first gray threshold to obtain the underwater image after the nonlinear filtering processing; calculating the second gray value and the second gray mean of the grayed surface image, and setting the second gray threshold based on the second gray mean, and performing weighted mean filtering of the grayed surface image using the weight coefficient of nonlinear distribution based on the second gray value and the second gray threshold to obtain the surface image after the nonlinear filtering processing.

[0061] Furthermore, the denoising process is performed on the water body parameters to obtain the water body parameters after denoising, including: performing wavelet decomposition on the water body parameters to obtain corresponding high-frequency wavelet coefficients and low-frequency wavelet coefficients; calculating the noise variance based on the high-frequency wavelet coefficients and low-frequency wavelet coefficients, scaling the high-frequency wavelet coefficients and low-frequency wavelet coefficients based on the noise variance to obtain the scaled high-frequency wavelet coefficients and low-frequency wavelet coefficients; performing inverse wavelet transformation based on the scaled high-frequency wavelet coefficients and low-frequency wavelet coefficients to obtain the denoised water body parameters.

[0062] Specifically, the underwater image and the surface image are grayed, and the grayed underwater image and the surface image are grayed according to the three primary color components of the underwater image and the surface image to obtain a grayed underwater image and a grayed surface image; a first gray value and a first gray mean of the grayed underwater image are calculated, the gray value of each grayed underwater pixel is calculated, and a first gray mean is calculated according to the gray value of each pixel, a first gray threshold is set based on the first gray mean, and a weighted mean filtering of the grayed underwater image is performed using a nonlinearly distributed weight coefficient based on the first gray value and the first gray threshold, and the expression of the nonlinearly distributed weight coefficient is:

[0063]

[0064] Among them, k i is the weight coefficient of nonlinear distribution, x i is the gray value of pixel i, x jis the gray value of the central pixel, d is the scale coefficient, T is the gray threshold, σ is the compensation coefficient, so as to obtain the underwater image after nonlinear filtering processing; the nonlinear filtering processing steps for the surface image are the same as those for the underwater image, which will not be repeated here; the water body parameters are subjected to wavelet decomposition, a mother wavelet function and a decomposition scale are selected, and the water body parameters are subjected to wavelet coefficient decomposition according to the mother wavelet function and the decomposition scale to obtain the corresponding high-frequency wavelet coefficients and low-frequency wavelet coefficients; based on the preset noise variance formula, the high-frequency wavelet coefficients and the low-frequency wavelet coefficients are used to obtain the corresponding high-frequency wavelet coefficients and low-frequency wavelet coefficients. The noise variance is calculated by wavelet coefficients; based on the noise variance, the high-frequency wavelet coefficients and the low-frequency wavelet coefficients are scaled, an estimator is calculated by the noise variance, a minimum value of the estimator is calculated by using a constraint factor, a scale threshold is calculated by the minimum value of the estimator, and the high-frequency wavelet coefficients and the low-frequency wavelet coefficients are scaled according to the scale threshold to obtain the high-frequency wavelet coefficients and the low-frequency wavelet coefficients after the scale transformation; based on the high-frequency wavelet coefficients and the low-frequency wavelet coefficients after the scale transformation, an inverse wavelet transform is performed, that is, coefficient reconstruction is performed to obtain the water body parameters after the noise reduction processing.

[0065] S14: The cloud platform determines the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and controls the underwater robot to perform dynamic tracking and refinement scanning on the aquatic target to obtain dynamic tracking and refinement scanning results;

[0066] In the specific implementation process of the present invention, the cloud platform determines the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and controls the underwater robot to perform dynamic tracking and refinement scanning on the aquatic target to obtain dynamic tracking and refinement scanning results, including: extracting the micro-Doppler spectrum of the water body parameters after noise reduction processing, and performing feature extraction based on the micro-Doppler spectrum to obtain micro-Doppler features; performing feature extraction on the underwater image after nonlinear filtering processing to obtain image semantic features; performing feature fusion of the micro-Doppler features and image semantic features to obtain target fusion features, and performing target recognition based on the target fusion features to determine the aquatic target; after determining the aquatic target, the cloud platform adjusts the moving path and scanning parameters of the underwater robot, and the underwater robot performs dynamic tracking and refinement scanning on the aquatic target based on the adjusted moving path and scanning parameters in combination with the LightCrafter4500 optical projector to obtain dynamic tracking and refinement scanning results.

[0067] Specifically, a micro-Doppler spectrum is extracted from the water body parameters after noise reduction processing, a short-time Fourier transform is performed on the water body parameters after noise reduction processing, and a short-time Fourier spectrum is obtained by using a sliding window function at the same time, and the micro-Doppler spectrum is extracted by using the short-time Fourier spectrum according to the autocorrelation item; feature extraction is performed based on the micro-Doppler spectrum, the maximum peak position and amplitude of the micro-Doppler spectrum are calculated, the maximum amplitude value sequence of the micro-Doppler spectrum is calculated, the inter-frame mean is calculated according to the maximum amplitude value sequence, the peak frequency and the peak frequency standard deviation are calculated according to the inter-frame mean, and the micro-Doppler feature is composed of the maximum peak position, amplitude, peak frequency and peak frequency standard deviation; feature extraction is performed on the underwater image after nonlinear filtering processing, the initial semantic features of the underwater image after nonlinear filtering processing are extracted, a semantic feature pyramid is constructed, semantic feature matching constraint rules are set based on geometric consistency, and an energy function is constructed, and the initial semantic features are matched in the semantic feature pyramid based on the semantic feature matching constraint rules and the energy function to obtain the final image semantic features; the micro-Doppler features and image semantic features are feature fused, and the conc The at function concatenates the micro-Doppler features and the image semantic features, and the concat concatenation function is a string-based function. The concatenated micro-Doppler features and image semantic features are then mapped to the desired target domain using a feature adapter to obtain a target fusion feature. Target recognition is performed based on the target fusion feature, and the target fusion feature is input into a target detection model for target recognition processing to determine the aquatic target. After determining the aquatic target, the cloud platform adjusts the movement path and scanning parameters of the underwater robot. The underwater robot dynamically tracks and refines the aquatic target based on the adjusted movement path and scanning parameters in combination with the LightCrafter4500 optical projector, that is, the aquatic target is tracked through a video acquisition system including the LightCrafter4500 optical projector and several sensors to obtain texture features, contours, sizes and other information of the aquatic target for refinement, and obtain dynamic tracking and refinement scanning results, thereby completing the intelligent processing of real-time dynamic water conditions, and being able to understand the real-time conditions of aquatic targets in the water area.

[0068] S15: The cloud platform calculates the optical flow of the water surface image using a motion estimation neural network based on the water surface image after nonlinear filtering.

[0069] In the specific implementation process of the present invention, the cloud platform calculates the optical flow of the water surface image based on the water surface image after nonlinear filtering processing using a motion estimation neural network, including: extracting features of the water surface image after nonlinear filtering processing to obtain a feature map; inputting the feature map into the motion estimation neural network, and calculating the optical flow of the water surface image based on the motion estimation neural network, wherein the motion estimation neural network is a convergence model obtained by inputting a sample data set into a deep neural network for training, and the deep neural network includes a channel attention mechanism module, a spatial attention module, an iterative update module and an upsampling module.

[0070] Specifically, feature extraction is performed on the water surface image after nonlinear filtering, an initial feature map is determined through the water surface image after nonlinear filtering, channel attention modulation is performed on the initial feature map, and the initial feature map after channel attention modulation is obtained. Convolution processing is performed on the initial feature map after channel attention modulation to obtain a convolution feature matrix, the convolution feature matrix is ​​converted into a channel feature vector, a combination matrix is ​​generated based on the channel feature vector, a secondary convolution is performed on the pixel points of the initial feature map after attention modulation to obtain a convolution matrix, a combination matrix is ​​generated based on the convolution matrix, the combination matrix and the combination matrix are multiplied to obtain a channel relationship matrix, and the initial feature map after channel attention modulation is processed based on the channel relationship matrix. Transformation, obtain a feature map; input the feature map into a motion estimation neural network, and calculate the optical flow of the water surface image based on the motion estimation neural network, wherein the motion estimation neural network is a convergence model obtained by inputting a sample data set into a deep neural network for training, and the deep neural network includes a channel attention mechanism module, a spatial attention module, an iterative update module, and an upsampling module. The channel attention mechanism module performs average pooling and maximum pooling on the input feature map to obtain different spatial context feature information, and generates a channel attention feature map by using a sigmoid activation function through a shared multi-layer perceptron for different spatial context feature information, and inputs the channel attention feature map into the spatial attention module. The spatial attention module performs maximum pooling and average pooling on the channel attention feature map again to obtain two different spatial feature maps, splices the two different spatial feature maps to obtain a spliced ​​feature map, convolves the spliced ​​feature map to obtain a spliced ​​feature map after convolution, generates a spatial attention feature map based on the spliced ​​feature map after convolution using a sigmoid activation function, and inputs the spatial attention feature map into the iterative update module. The iterative update module performs iterative update calculations of the optical flow of the water surface image through the spatial attention feature map, and the upsampling module is used to upsample the optical flow of the water surface image calculated by the iterative update. Because the movement of fish during feeding time will generate optical flow, the calculation of the optical flow of the water surface image through the motion estimation network can reflect the feeding intensity of fish. The use of optical flow enables the system to evaluate the intensity of fish feeding and helps detect overfeeding to help regulate the feeding process.

[0071] S16: constructing a three-dimensional model of the water environment based on the underwater image after nonlinear filtering, the water parameters after noise reduction and the dynamic tracking and refinement scanning results, and establishing a decision model based on the three-dimensional model of the water environment and the optical flow of the water surface image.

[0072] In the specific implementation process of the present invention, a three-dimensional model of a water environment is constructed based on the underwater image after nonlinear filtering processing, the water body parameters after noise reduction processing and the dynamic tracking and refinement scanning results, and a decision model is established based on the underwater three-dimensional model and the optical flow of the water surface image, including: extracting modeling parameters based on the underwater image after nonlinear filtering processing, the water body parameters after noise reduction processing and the dynamic tracking and refinement scanning results, and constructing a three-dimensional model of a water environment based on the modeling parameters using the digital twin technology; performing visual simulation based on the three-dimensional model of the water environment in combination with the digital twin to obtain visual simulation results, and obtaining global data information based on the visual simulation results and the optical flow of the water surface image, and establishing a decision model based on the global data information.

[0073] Specifically, modeling parameters are extracted based on underwater images after nonlinear filtering processing, water body parameters after noise reduction processing and dynamic tracking and refinement scanning results, such as coordinate position information of the water environment, types and quantity of aquatic targets, etc. A three-dimensional model of the water environment is constructed based on the modeling parameters using digital twin technology, an initial three-dimensional model of the water environment is constructed using the modeling parameters, and finite element simulations are performed on the initial three-dimensional model multiple times in combination with the types of aquatic targets and changes in water body parameters to obtain overall simulation data of the water environment, a state differential simulator is solved based on the overall simulation data of the water environment, a digital twin model of the water environment is constructed based on the state differential simulator, and the three-dimensional model of the water environment is mapped in a multi-graphics processor distributed computing environment using the state differential simulator using the overall simulation data and modeling parameters, which is the three-dimensional model of the water environment, digital twin technology The technology is to map and synchronize with physical entities through various information technology means to achieve real-time monitoring, analysis and simulation of physical objects; based on the three-dimensional model of the water environment combined with the digital twin for visual simulation, the digital twin is used to simulate the required actual information of the three-dimensional model of the water environment, so that it has stronger simulation space and time characteristics. The three-dimensional model of the water environment can be used through the digital twin to realize intelligent simulation of physical objects and obtain visual simulation results. Based on the visual simulation results and the optical flow of the water surface image, global data information is obtained. The global data information is the biological information and environmental information of the entire breeding area. A decision-making model is established based on the global data information. The breeding adjustment strategy for fish and other organisms in the area and the regulation strategy of the water environment can be obtained through the decision-making model, so as to realize scientific and intelligent decision-making in aquatic areas.

[0074] In an embodiment of the present invention, the error of the inertial navigation system is continuously corrected through a dynamic obstacle avoidance algorithm and GPS positioning technology to continuously obtain accurate position parameters, so that the local path can be adjusted in time to avoid deviation from the operation target point, and nonlinear filtering processing of the image is performed using nonlinearly distributed weight coefficients to make the obtained image clearer. At the same time, wavelet transform is used to perform noise reduction processing on water parameters to make the obtained water parameters more accurate. The optical flow of the water surface image is calculated through a motion estimation network to reflect the feeding intensity of fish. At the same time, digital twin technology is used to establish a three-dimensional model of the water environment, so that the established three-dimensional model of the water environment can more accurately reflect the actual state of the aquaculture area. Combining the three-dimensional model of the water environment and the optical flow of the water surface image can establish a decision-making model to achieve scientific and intelligent decision-making in aquatic areas.

[0075] Embodiment 2

[0076] See also Figure 2 , Figure 2 It is a schematic diagram of the structural composition of a water body three-dimensional modeling device based on an underwater robot in an embodiment of the present invention.

[0077] like Figure 2 As shown, a water body three-dimensional modeling device based on an underwater robot, the device comprises:

[0078] The underwater robot moving module 21 is used to start the underwater robot, and the underwater robot moves to each preset operation target point based on the preset operation path by using the inertial navigation system, dynamic obstacle avoidance algorithm and GPS positioning technology;

[0079] In the specific implementation process of the present invention, the underwater robot uses an inertial navigation system, a dynamic obstacle avoidance algorithm and GPS positioning technology to move to each preset operation target point, including: when the underwater robot moves based on a preset operation path, the initial current position parameters of the underwater robot are calculated based on the inertial navigation system; the initial current position parameters are error corrected based on the GPS positioning technology to obtain corrected position parameters; local path planning is performed based on the corrected position parameters using a dynamic obstacle avoidance algorithm to obtain a local operation path, and the preset operation path is adjusted based on the local operation path.

[0080] Specifically, the user can start the underwater robot through the terminal device and set the operation path of the underwater robot. When the underwater robot moves based on the preset operation path, the inertial navigation system belongs to the dead reckoning navigation mode, that is, the position of the next point is calculated from the position of a known point according to the continuously measured heading angle and speed of the moving body, so the initial current position parameters of the underwater robot can be continuously measured. However, since the navigation information is generated through integration, its positioning error increases with time; therefore, based on the Global Positioning System (GPS), the positioning error of the underwater robot is gradually increased. System, GPS) positioning technology is used to perform error correction on the initial current position parameters. The GPS positioning technology can provide an accurate geographic spatial position through a high-precision positioning system. The position parameters calculated by the inertial navigation system are corrected by the GPS positioning technology to obtain the corrected position parameters, which are the accurate actual position parameters of the underwater robot. Although the position parameters have been corrected and the accurate actual position parameters of the underwater robot are known, the actual position of the underwater robot has shifted. Based on the corrected position parameters, a dynamic obstacle avoidance algorithm is used to perform local path planning, with the current position point as the starting point and the corresponding next point in the preset operation path as the target point. A plurality of path nodes are set according to the starting point and the target point, and the cost value of the target point is set to zero, and the cost value of all path nodes is set to zero. The cost value is set to infinity, starting from the target point, searching for a path to the starting point according to the heuristic rule, updating the path cost value according to the cost value of the neighboring nodes of the current path node, tracing back from the target point to the starting point along the updated path, updating the cost value of the neighboring nodes of each path node on the path, in each update, detecting whether the change of the cost value of the path node exceeds the preset threshold, if it exceeds the preset threshold, searching for another path node, repeating the above process until the preset number of iterations is reached, and obtaining the local operation path, adjusting the preset operation path based on the local operation path, replacing the corresponding part of the operation path with the local operation path, thereby adjusting the preset operation path so that the underwater robot can continue to move along each preset operation target point.

[0081] Image and parameter acquisition module 22: used for the underwater robot to obtain underwater images and surface images at the preset operation target point using a video acquisition system, obtain water body parameters using a plurality of sensors, and transmit the underwater images, surface images and water body parameters to a cloud platform;

[0082] In the specific implementation process of the present invention, the transmitting of the underwater image, the surface image and the water body parameters to the cloud platform includes: transmitting the water body parameters, the surface image and the underwater image to the cloud platform using a fuzzy control algorithm based on a wireless sensor network.

[0083] Specifically, the underwater robot uses a video acquisition system to obtain underwater images and surface images at each preset operation target point. The video acquisition system uses a charge coupled device (CDD) industrial camera and a LightCrafter 4500 optical projector, and uses a number of sensors to obtain water parameters. The water parameters include signal parameters and temperature parameters, etc. The several sensors include dissolved oxygen sensors, ammonia nitrogen sensors, ultrasonic ranging sensors, pH sensors, temperature sensors, turbidity sensors, and gravity sensors, etc. Based on a wireless sensor network, a fuzzy control algorithm is used to transmit the water parameters, surface images, and underwater images to a cloud platform, and the water parameters, surface images, and underwater images are compressed and packaged to obtain packet data, and a delay rate of packet data transmission is calculated. The delay rate is fuzzy controlled by a fuzzy control algorithm, that is, the delay rate is fuzzified to obtain a set of precise quantities of transmission delay and a set of fuzzy quantities corresponding to the precise quantities, and the calculation is performed. The membership between the set of precise quantities of transmission delay and the set of fuzzy quantities corresponding to the precise quantities is obtained according to the membership, and the fuzzy subset is obtained according to the membership. The fuzzy control output is inferred by the fuzzy subset using fuzzy control logic, and the fuzzy control output is defuzzified to obtain the corresponding fuzzy control result. The fuzzy control result includes a suitable delay and a transfer function. The fuzzy control algorithm is used to solve the problem of data lag, so that data transmission is faster and smoother; according to the fuzzy control result, the packet data is transmitted to the cloud platform by using a wireless sensor network. The wireless sensor network is a wireless network composed of a large number of sensors in a self-organizing and multi-hop manner, which senses, collects, processes and transmits information of sensed objects in the geographical area covered by the network in a collaborative manner.

[0084] Image and parameter processing module 23: used for the cloud platform to perform nonlinear filtering on the underwater image and the surface image to obtain the underwater image and the surface image after the nonlinear filtering, and to perform noise reduction on the water body parameters to obtain the water body parameters after the noise reduction;

[0085] In the specific implementation process of the present invention, the underwater image and the surface image are subjected to nonlinear filtering processing to obtain the underwater image and the surface image after the nonlinear filtering processing, including: graying the underwater image and the surface image to obtain the grayed underwater image and the grayed surface image; calculating the first gray value and the first gray mean of the grayed underwater image, and setting the first gray threshold based on the first gray mean, and performing weighted mean filtering of the grayed underwater image using the weight coefficient of nonlinear distribution based on the first gray value and the first gray threshold to obtain the underwater image after the nonlinear filtering processing; calculating the second gray value and the second gray mean of the grayed surface image, and setting the second gray threshold based on the second gray mean, and performing weighted mean filtering of the grayed surface image using the weight coefficient of nonlinear distribution based on the second gray value and the second gray threshold to obtain the surface image after the nonlinear filtering processing.

[0086] Furthermore, the denoising process is performed on the water body parameters to obtain the water body parameters after denoising, including: performing wavelet decomposition on the water body parameters to obtain corresponding high-frequency wavelet coefficients and low-frequency wavelet coefficients; calculating the noise variance based on the high-frequency wavelet coefficients and low-frequency wavelet coefficients, scaling the high-frequency wavelet coefficients and low-frequency wavelet coefficients based on the noise variance to obtain the scaled high-frequency wavelet coefficients and low-frequency wavelet coefficients; performing inverse wavelet transformation based on the scaled high-frequency wavelet coefficients and low-frequency wavelet coefficients to obtain the denoised water body parameters.

[0087] Specifically, the underwater image and the surface image are grayed, and the grayed underwater image and the surface image are grayed according to the three primary color components of the underwater image and the surface image to obtain a grayed underwater image and a grayed surface image; a first gray value and a first gray mean of the grayed underwater image are calculated, the gray value of each grayed underwater pixel is calculated, and a first gray mean is calculated according to the gray value of each pixel, a first gray threshold is set based on the first gray mean, and a weighted mean filtering of the grayed underwater image is performed using a nonlinearly distributed weight coefficient based on the first gray value and the first gray threshold, and the expression of the nonlinearly distributed weight coefficient is:

[0088]

[0089] Among them, k i is the weight coefficient of nonlinear distribution, x i is the gray value of pixel i, x jis the gray value of the central pixel, d is the scale coefficient, T is the gray threshold, σ is the compensation coefficient, so as to obtain the underwater image after nonlinear filtering processing; the nonlinear filtering processing steps for the surface image are the same as those for the underwater image, which will not be repeated here; the water body parameters are subjected to wavelet decomposition, a mother wavelet function and a decomposition scale are selected, and the water body parameters are subjected to wavelet coefficient decomposition according to the mother wavelet function and the decomposition scale to obtain the corresponding high-frequency wavelet coefficients and low-frequency wavelet coefficients; based on the preset noise variance formula, the high-frequency wavelet coefficients and the low-frequency wavelet coefficients are used to obtain the corresponding high-frequency wavelet coefficients and low-frequency wavelet coefficients. The noise variance is calculated by wavelet coefficients; based on the noise variance, the high-frequency wavelet coefficients and the low-frequency wavelet coefficients are scaled, an estimator is calculated by the noise variance, a minimum value of the estimator is calculated by using a constraint factor, a scale threshold is calculated by the minimum value of the estimator, and the high-frequency wavelet coefficients and the low-frequency wavelet coefficients are scaled according to the scale threshold to obtain the high-frequency wavelet coefficients and the low-frequency wavelet coefficients after the scale transformation; based on the high-frequency wavelet coefficients and the low-frequency wavelet coefficients after the scale transformation, an inverse wavelet transform is performed, that is, coefficient reconstruction is performed to obtain the water body parameters after the noise reduction processing.

[0090] Tracking and refining scanning module 24: used for the cloud platform to determine the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and control the underwater robot to perform dynamic tracking and refining scanning on the aquatic target to obtain dynamic tracking and refining scanning results;

[0091] In the specific implementation process of the present invention, the cloud platform determines the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and controls the underwater robot to perform dynamic tracking and refinement scanning on the aquatic target to obtain dynamic tracking and refinement scanning results, including: extracting the micro-Doppler spectrum of the water body parameters after noise reduction processing, and performing feature extraction based on the micro-Doppler spectrum to obtain micro-Doppler features; performing feature extraction on the underwater image after nonlinear filtering processing to obtain image semantic features; performing feature fusion of the micro-Doppler features and image semantic features to obtain target fusion features, and performing target recognition based on the target fusion features to determine the aquatic target; after determining the aquatic target, the cloud platform adjusts the moving path and scanning parameters of the underwater robot, and the underwater robot performs dynamic tracking and refinement scanning on the aquatic target based on the adjusted moving path and scanning parameters in combination with the LightCrafter4500 optical projector to obtain dynamic tracking and refinement scanning results.

[0092] Specifically, a micro-Doppler spectrum is extracted from the water body parameters after noise reduction processing, a short-time Fourier transform is performed on the water body parameters after noise reduction processing, and a short-time Fourier spectrum is obtained by using a sliding window function at the same time, and the micro-Doppler spectrum is extracted by using the short-time Fourier spectrum according to the autocorrelation item; feature extraction is performed based on the micro-Doppler spectrum, the maximum peak position and amplitude of the micro-Doppler spectrum are calculated, the maximum amplitude value sequence of the micro-Doppler spectrum is calculated, the inter-frame mean is calculated according to the maximum amplitude value sequence, the peak frequency and the peak frequency standard deviation are calculated according to the inter-frame mean, and the micro-Doppler feature is composed of the maximum peak position, amplitude, peak frequency and peak frequency standard deviation; feature extraction is performed on the underwater image after nonlinear filtering processing, the initial semantic features of the underwater image after nonlinear filtering processing are extracted, a semantic feature pyramid is constructed, semantic feature matching constraint rules are set based on geometric consistency, and an energy function is constructed, and the initial semantic features are matched in the semantic feature pyramid based on the semantic feature matching constraint rules and the energy function to obtain the final image semantic features; the micro-Doppler features and image semantic features are feature fused, and the conc The at function concatenates the micro-Doppler features and the image semantic features, and the concat concatenation function is a string-based function. The concatenated micro-Doppler features and image semantic features are then mapped to the desired target domain using a feature adapter to obtain a target fusion feature. Target recognition is performed based on the target fusion feature, and the target fusion feature is input into a target detection model for target recognition processing to determine the aquatic target. After determining the aquatic target, the cloud platform adjusts the movement path and scanning parameters of the underwater robot. The underwater robot dynamically tracks and refines the aquatic target based on the adjusted movement path and scanning parameters in combination with the LightCrafter4500 optical projector, that is, the aquatic target is tracked through a video acquisition system including the LightCrafter4500 optical projector and several sensors to obtain texture features, contours, sizes and other information of the aquatic target for refinement, and obtain dynamic tracking and refinement scanning results, thereby completing the intelligent processing of real-time dynamic water conditions, and being able to understand the real-time conditions of aquatic targets in the water area.

[0093] Water surface image optical flow module 25: used for the cloud platform to calculate the water surface image optical flow using a motion estimation neural network based on the water surface image after nonlinear filtering;

[0094] In the specific implementation process of the present invention, the cloud platform calculates the optical flow of the water surface image based on the water surface image after nonlinear filtering processing using a motion estimation neural network, including: extracting features of the water surface image after nonlinear filtering processing to obtain a feature map; inputting the feature map into the motion estimation neural network, and calculating the optical flow of the water surface image based on the motion estimation neural network, wherein the motion estimation neural network is a convergence model obtained by inputting a sample data set into a deep neural network for training, and the deep neural network includes a channel attention mechanism module, a spatial attention module, an iterative update module and an upsampling module.

[0095] Specifically, feature extraction is performed on the water surface image after nonlinear filtering, an initial feature map is determined through the water surface image after nonlinear filtering, channel attention modulation is performed on the initial feature map, and the initial feature map after channel attention modulation is obtained. Convolution processing is performed on the initial feature map after channel attention modulation to obtain a convolution feature matrix, the convolution feature matrix is ​​converted into a channel feature vector, a combination matrix is ​​generated based on the channel feature vector, a secondary convolution is performed on the pixel points of the initial feature map after attention modulation to obtain a convolution matrix, a combination matrix is ​​generated based on the convolution matrix, the combination matrix and the combination matrix are multiplied to obtain a channel relationship matrix, and the initial feature map after channel attention modulation is processed based on the channel relationship matrix. Transformation, obtain a feature map; input the feature map into a motion estimation neural network, and calculate the optical flow of the water surface image based on the motion estimation neural network, wherein the motion estimation neural network is a convergence model obtained by inputting a sample data set into a deep neural network for training, and the deep neural network includes a channel attention mechanism module, a spatial attention module, an iterative update module, and an upsampling module. The channel attention mechanism module performs average pooling and maximum pooling on the input feature map to obtain different spatial context feature information, and generates a channel attention feature map by using a sigmoid activation function through a shared multi-layer perceptron for different spatial context feature information, and inputs the channel attention feature map into the spatial attention module. The spatial attention module performs maximum pooling and average pooling on the channel attention feature map again to obtain two different spatial feature maps, splices the two different spatial feature maps to obtain a spliced ​​feature map, convolves the spliced ​​feature map to obtain a spliced ​​feature map after convolution, generates a spatial attention feature map based on the spliced ​​feature map after convolution using a sigmoid activation function, and inputs the spatial attention feature map into the iterative update module. The iterative update module performs iterative update calculations of the optical flow of the water surface image through the spatial attention feature map, and the upsampling module is used to upsample the optical flow of the water surface image calculated by the iterative update. Because the movement of fish during feeding time will generate optical flow, the calculation of the optical flow of the water surface image through the motion estimation network can reflect the feeding intensity of fish. The use of optical flow enables the system to evaluate the intensity of fish feeding and helps detect overfeeding to help regulate the feeding process.

[0096] The three-dimensional modeling module 26 is used to construct a three-dimensional model of the water environment based on the underwater image after nonlinear filtering, the water parameters after noise reduction and the dynamic tracking and refinement scanning results, and to establish a decision model based on the three-dimensional model of the water environment and the optical flow of the water surface image.

[0097] In the specific implementation process of the present invention, a three-dimensional model of a water environment is constructed based on the underwater image after nonlinear filtering processing, the water body parameters after noise reduction processing and the dynamic tracking and refinement scanning results, and a decision model is established based on the underwater three-dimensional model and the optical flow of the water surface image, including: extracting modeling parameters based on the underwater image after nonlinear filtering processing, the water body parameters after noise reduction processing and the dynamic tracking and refinement scanning results, and constructing a three-dimensional model of a water environment based on the modeling parameters using the digital twin technology; performing visual simulation based on the three-dimensional model of the water environment in combination with the digital twin to obtain visual simulation results, and obtaining global data information based on the visual simulation results and the optical flow of the water surface image, and establishing a decision model based on the global data information.

[0098] Specifically, modeling parameters are extracted based on underwater images after nonlinear filtering processing, water body parameters after noise reduction processing and dynamic tracking and refinement scanning results, such as coordinate position information of the water environment, types and quantity of aquatic targets, etc. A three-dimensional model of the water environment is constructed based on the modeling parameters using digital twin technology, an initial three-dimensional model of the water environment is constructed using the modeling parameters, and finite element simulations are performed on the initial three-dimensional model multiple times in combination with the types of aquatic targets and changes in water body parameters to obtain overall simulation data of the water environment, a state differential simulator is solved based on the overall simulation data of the water environment, a digital twin model of the water environment is constructed based on the state differential simulator, and the three-dimensional model of the water environment is mapped in a multi-graphics processor distributed computing environment using the state differential simulator using the overall simulation data and modeling parameters, which is the three-dimensional model of the water environment, digital twin technology The technology is to map and synchronize with physical entities through various information technology means to achieve real-time monitoring, analysis and simulation of physical objects; based on the three-dimensional model of the water environment combined with the digital twin for visual simulation, the digital twin is used to simulate the required actual information of the three-dimensional model of the water environment, so that it has stronger simulation space and time characteristics. The three-dimensional model of the water environment can be used through the digital twin to realize intelligent simulation of physical objects and obtain visual simulation results. Based on the visual simulation results and the optical flow of the water surface image, global data information is obtained. The global data information is the biological information and environmental information of the entire breeding area. A decision-making model is established based on the global data information. The breeding adjustment strategy for fish and other organisms in the area and the regulation strategy of the water environment can be obtained through the decision-making model, so as to realize scientific and intelligent decision-making in aquatic areas.

[0099] In an embodiment of the present invention, the error of the inertial navigation system is continuously corrected through a dynamic obstacle avoidance algorithm and GPS positioning technology to continuously obtain accurate position parameters, so that the local path can be adjusted in time to avoid deviation from the operation target point, and nonlinear filtering processing of the image is performed using nonlinearly distributed weight coefficients to make the obtained image clearer. At the same time, wavelet transform is used to perform noise reduction processing on water parameters to make the obtained water parameters more accurate. The optical flow of the water surface image is calculated through a motion estimation network to reflect the feeding intensity of fish. At the same time, digital twin technology is used to establish a three-dimensional model of the water environment, so that the established three-dimensional model of the water environment can more accurately reflect the actual state of the aquaculture area. Combining the three-dimensional model of the water environment and the optical flow of the water surface image can establish a decision-making model to achieve scientific and intelligent decision-making in aquatic areas.

[0100] Embodiment 3

[0101] See also Figure 3 , Figure 3 It is a schematic diagram of the structural composition of a water body three-dimensional modeling system based on an underwater robot in an embodiment of the present invention.

[0102] like Figure 3 As shown, a water body three-dimensional modeling system based on an underwater robot, the system includes an underwater robot and a cloud platform 37, the underwater robot includes a control mainboard 31, a power supply 32, a sensor 33, a camera 34, a wireless communication system 35 and a motion control system 36, wherein the control mainboard 31 is connected to the power supply 32, the sensor 33, the camera 34, the wireless communication system 35 and the motion control system 36 respectively, the cloud platform 37 establishes a communication connection with the wireless communication system 35 of the underwater robot, and the system is configured to perform the above-mentioned water body three-dimensional modeling method based on the underwater robot.

[0103] In the specific implementation process of the present invention, the control mainboard 31 is used to control the underwater robot as a whole, the power supply 32 is used to provide power for the underwater robot, the sensor 33 includes a dissolved oxygen sensor, an ammonia nitrogen sensor, an ultrasonic distance sensor, a pH sensor, a temperature sensor, a turbidity sensor and a gravity sensor, etc., the camera 34 includes a CCD industrial camera and a LightCrafter 4500 optical projector, the motion control system 36 includes an inertial navigation system and a GPS positioning system, and the wireless communication system 35 is used to transmit data through a communication connection with a cloud platform 37. The cloud platform 37 includes an aquaculture cloud, which is a private storage cloud for storing relevant data for big data analysis. Figure 3 The system shown does not constitute a limitation of all components, and may include more or less components than shown, or combine certain components.

[0104] In the specific implementation process of the present invention, the specific implementation methods of the system items can be referred to the above embodiments, which will not be repeated here.

[0105] In an embodiment of the present invention, the error of the inertial navigation system is continuously corrected through a dynamic obstacle avoidance algorithm and GPS positioning technology to continuously obtain accurate position parameters, so that the local path can be adjusted in time to avoid deviation from the operation target point, and nonlinear filtering processing of the image is performed using nonlinearly distributed weight coefficients to make the obtained image clearer. At the same time, wavelet transform is used to perform noise reduction processing on water parameters to make the obtained water parameters more accurate. The optical flow of the water surface image is calculated through a motion estimation network to reflect the feeding intensity of fish. At the same time, digital twin technology is used to establish a three-dimensional model of the water environment, so that the established three-dimensional model of the water environment can more accurately reflect the actual state of the aquaculture area. Combining the three-dimensional model of the water environment and the optical flow of the water surface image can establish a decision-making model to achieve scientific and intelligent decision-making in aquatic areas.

[0106] A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0107] In addition, the above is a detailed introduction to a method, device and system for three-dimensional modeling of water bodies based on an underwater robot provided in an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for three-dimensional modeling of water bodies based on an underwater robot, characterized in that: The method comprises: Starting the underwater robot, which moves to each preset operation target point based on a preset operation path using an inertial navigation system, a dynamic obstacle avoidance algorithm, and GPS positioning technology; The underwater robot uses a video acquisition system to obtain underwater images and surface images at the preset operation target point, uses a plurality of sensors to obtain water body parameters, and transmits the underwater images, surface images and water body parameters to a cloud platform; The cloud platform performs nonlinear filtering on the underwater image and the surface image to obtain the underwater image and the surface image after the nonlinear filtering, and performs noise reduction on the water body parameters to obtain the water body parameters after the noise reduction; The cloud platform determines the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and controls the underwater robot to perform dynamic tracking and refinement scanning on the aquatic target to obtain dynamic tracking and refinement scanning results; The cloud platform calculates the optical flow of the water surface image using a motion estimation neural network based on the water surface image after nonlinear filtering; A three-dimensional model of the water environment is constructed based on the underwater image after nonlinear filtering, the water parameters after noise reduction and the dynamic tracking and refinement scanning results, and a decision model is established based on the three-dimensional model of the water environment and the optical flow of the water surface image.

2. The method for three-dimensional modeling of a water body based on an underwater robot according to claim 1, characterized in that: The underwater robot uses an inertial navigation system, a dynamic obstacle avoidance algorithm and GPS positioning technology to move to each preset operation target point, including: When the underwater robot moves based on a preset operation path, initial current position parameters of the underwater robot are calculated based on the inertial navigation system; Performing error correction on the initial current position parameter based on GPS positioning technology to obtain a corrected position parameter; Based on the corrected position parameters, a dynamic obstacle avoidance algorithm is used to perform local path planning to obtain a local operating path, and the preset operating path is adjusted based on the local operating path.

3. The method for three-dimensional modeling of a water body based on an underwater robot according to claim 1, characterized in that: The transmitting the underwater image, the surface image and the water body parameters to the cloud platform comprises: The water body parameters, water surface images and underwater images are transmitted to a cloud platform using a fuzzy control algorithm based on a wireless sensor network.

4. The method for three-dimensional modeling of a water body based on an underwater robot according to claim 1, characterized in that: The performing nonlinear filtering on the underwater image and the surface image to obtain the underwater image and the surface image after the nonlinear filtering includes: Performing grayscale processing on the underwater image and the water surface image to obtain a grayscale underwater image and a grayscale water surface image; Calculating a first grayscale value and a first grayscale mean of the grayscale underwater image, setting a first grayscale threshold based on the first grayscale mean, and performing weighted mean filtering on the grayscale underwater image using a weight coefficient of a nonlinear distribution based on the first grayscale value and the first grayscale threshold to obtain an underwater image after nonlinear filtering; Calculate the second grayscale value and the second grayscale mean of the grayscale water surface image, set the second grayscale threshold based on the second grayscale mean, perform weighted mean filtering on the grayscale water surface image using nonlinearly distributed weight coefficients based on the second grayscale value and the second grayscale threshold, and obtain the water surface image after nonlinear filtering.

5. The method for three-dimensional modeling of water body based on underwater robot according to claim 1, characterized in that: The performing noise reduction processing on the water body parameters to obtain the water body parameters after noise reduction processing includes: Performing wavelet decomposition on the water body parameters to obtain corresponding high-frequency wavelet coefficients and low-frequency wavelet coefficients; Calculating the noise variance based on the high-frequency wavelet coefficients and the low-frequency wavelet coefficients, and scaling the high-frequency wavelet coefficients and the low-frequency wavelet coefficients based on the noise variance to obtain scale-transformed high-frequency wavelet coefficients and low-frequency wavelet coefficients; Based on the high-frequency wavelet coefficients and low-frequency wavelet coefficients after scale transformation, wavelet inverse transform is performed to obtain the water body parameters after noise reduction processing.

6. The method for three-dimensional modeling of a water body based on an underwater robot according to claim 1, characterized in that: The cloud platform determines the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and controls the underwater robot to perform dynamic tracking and refinement scanning on the aquatic target to obtain dynamic tracking and refinement scanning results, including: Extracting a micro-Doppler spectrum of the water body parameters after noise reduction processing, and performing feature extraction based on the micro-Doppler spectrum to obtain a micro-Doppler feature; Perform feature extraction on underwater images after nonlinear filtering to obtain image semantic features; The micro-Doppler feature and the image semantic feature are subjected to feature fusion to obtain a target fusion feature, and target recognition is performed based on the target fusion feature to determine the aquatic target; After determining the aquatic target, the cloud platform adjusts the moving path and scanning parameters of the underwater robot. The underwater robot performs dynamic tracking and detailed scanning on the aquatic target based on the adjusted moving path and scanning parameters in combination with the LightCrafter4500 optical projector to obtain dynamic tracking and detailed scanning results.

7. The method for three-dimensional modeling of a water body based on an underwater robot according to claim 1, characterized in that: The cloud platform calculates the optical flow of the water surface image using a motion estimation neural network based on the water surface image after nonlinear filtering, including: Perform feature extraction on the water surface image after nonlinear filtering to obtain a feature map; The feature map is input into a motion estimation neural network, and the optical flow of the water surface image is calculated based on the motion estimation neural network, wherein the motion estimation neural network is a convergence model obtained by inputting a sample data set into a deep neural network for training, and the deep neural network includes a channel attention mechanism module, a spatial attention module, an iterative update module and an upsampling module.

8. The method for three-dimensional modeling of water bodies based on underwater robots according to claim 1, characterized in that: The method comprises: constructing a three-dimensional model of the water environment based on the underwater image after nonlinear filtering, the water parameters after noise reduction processing and the dynamic tracking and refinement scanning results, and establishing a decision model based on the three-dimensional model of the water environment and the optical flow of the water surface image, including: Extracting modeling parameters based on underwater images after nonlinear filtering, water body parameters after noise reduction, and dynamic tracking and refinement scanning results, and constructing a three-dimensional model of the water environment using digital twin technology based on the modeling parameters; Based on the three-dimensional model of the water environment combined with the digital twin, a visual simulation is performed to obtain a visual simulation result, and global data information is obtained based on the visual simulation result and the optical flow of the water surface image, and a decision model is established based on the global data information.

9. A three-dimensional water modeling device based on an underwater robot, characterized in that: The device comprises: Underwater robot moving module: used to start the underwater robot, which moves to each preset operation target point based on the preset operation path using an inertial navigation system, a dynamic obstacle avoidance algorithm and GPS positioning technology; Image and parameter acquisition module: used for the underwater robot to obtain underwater images and surface images at the preset operation target point using a video acquisition system, obtain water body parameters using a number of sensors, and transmit the underwater images, surface images and water body parameters to the cloud platform; Image and parameter processing module: used for the cloud platform to perform nonlinear filtering on the underwater image and the surface image to obtain the underwater image and the surface image after the nonlinear filtering, and to perform noise reduction on the water body parameters to obtain the water body parameters after the noise reduction; Tracking and refining scanning module: used for the cloud platform to determine the aquatic target based on the water body parameters after noise reduction processing and the underwater image after nonlinear filtering processing, and control the underwater robot to perform dynamic tracking and refining scanning on the aquatic target to obtain dynamic tracking and refining scanning results; Water surface image optical flow module: used for the cloud platform to calculate the water surface image optical flow using a motion estimation neural network based on the water surface image after nonlinear filtering; 3D modeling module: used to build a 3D model of the water environment based on the underwater image after nonlinear filtering, the water parameters after noise reduction and the dynamic tracking and refinement scanning results, and to establish a decision model based on the 3D model of the water environment and the optical flow of the water surface image.

10. A water body three-dimensional modeling system based on an underwater robot, characterized in that: The system includes an underwater robot and a cloud platform, wherein the underwater robot includes a control mainboard, a power supply, a sensor, a camera, a wireless communication system and a motion control system, wherein the control mainboard is connected to the power supply, the sensor, the camera, the wireless communication system and the motion control system respectively, and the cloud platform establishes a communication connection with the wireless communication system of the underwater robot, and the system is configured to execute the three-dimensional water body modeling method based on the underwater robot according to any one of claims 1-8.

Images