Cluster control system and method based on bionic intelligent fish equipment, and storage medium

Abstract

This application provides a cluster control system, method, and storage medium based on a bionic intelligent fish device. The system includes: a visual perception system that performs three-dimensional reconstruction of the environment based on image data; identifies obstacles in the environment and measures the distance to them in real time to determine the distance between the bionic intelligent fish device and the obstacle; performs environmental mapping and path planning optimization based on point cloud data, obtains the position and posture of the bionic intelligent fish device in real time, and adjusts the position and posture of the bionic intelligent fish device; a path planning system that generates an optimal path from a starting point to a target point for the bionic intelligent fish device based on the three-dimensional reconstructed environment; a bionic propulsion system with a streamlined design that mimics the bionic appearance of a fish; a tail fin made of composite materials and a tail fin drive system; the tail fin drive system drives the tail fin to simulate the swinging pattern of a fish's tail fin, propelling the bionic intelligent fish device to move. This application improves the smooth and efficient movement of bionic intelligent fish devices in complex environments.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of bionic intelligent underwater robots, and in particular to an intelligent bionic fish navigation and decision-making system based on double-layer feedback control. Background Art

[0002] With the rapid development of the marine economy and technology, traditional monitoring methods are no longer able to meet increasingly complex needs. The environmental complexity of underwater missions, the high precision requirements, and the need for real-time responsiveness pose numerous challenges to existing technologies. The design and application of bionic fish, as flexible and low-noise underwater robots, have become a research hotspot in this field. However, existing bionic fish suffer from numerous challenges, such as limited perception, inefficient path planning, and insufficient environmental adaptability. In particular, their autonomy and intelligence in complex environments still need to be improved.

[0003] Existing path planning technologies rely heavily on static algorithms and local obstacle avoidance designs, lacking the ability to cope with dynamic obstacles and real-time environmental changes. Furthermore, most bionic fish systems employ relatively simple control schemes, failing to fully exploit their advantages in complex dynamic environments, resulting in low task execution efficiency and poor adaptability. To address this issue, a smart bionic fish navigation and decision-making system based on two-layer feedback control is proposed. By integrating visual perception, intelligent path planning, bionic propulsion, and Internet of Things technologies, this system significantly improves the bionic fish's intelligence and the efficiency of its underwater operations. Summary of the Invention

[0004] The present application provides a cluster control system, method, and storage medium based on bionic intelligent fish equipment to at least solve the above technical problems existing in the prior art.

[0005] According to a first aspect of the present application, a cluster control system based on a bionic intelligent fish device is provided, the system comprising a visual perception system, a path planning system, a bionic propulsion system and an Internet of Things communication system; wherein:

[0006] The visual perception system is used to obtain image data of the surrounding environment collected by the bionic intelligent fish device and perform three-dimensional reconstruction of the environment based on the image data; identify obstacles in the environment and measure the distance between the obstacles in real time to determine the distance between the bionic intelligent fish device and the obstacles to assist the bionic intelligent fish device in moving in a dynamic environment; use the laser radar on the bionic intelligent fish device to generate point cloud data for environmental mapping and path planning optimization, and use the inertial measurement unit on the bionic intelligent fish device to collect the device's posture and motion state to obtain the bionic intelligent fish device's position and posture in real time, so as to adjust the bionic intelligent fish device's position and posture to ensure that the bionic intelligent fish device maintains stable operation in a complex environment;

[0007] A path planning system is used to generate the optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment. The path is adjusted in real time to avoid obstacles and ensure the safe movement of the bionic intelligent fish device.

[0008] The bionic propulsion system has a streamlined design and is designed to mimic the fish's bionic appearance. It is equipped with a composite tail fin and a tail fin drive system. The tail fin drive system drives the tail fin to simulate the swinging pattern of the fish's tail fin, propelling the bionic intelligent fish device through the water.

[0009] The Internet of Things communication system is used to build data transmission links between bionic intelligent fish devices and between bionic intelligent fish devices and cloud servers, enabling data interaction between bionic intelligent fish devices and between bionic intelligent fish devices and cloud servers to ensure the transmission of perception data, status information and task instructions.

[0010] In some optional embodiments, the visual perception system includes:

[0011] The visual sensing module is used to collect image data of the surrounding environment, identify surrounding objects, targets, and potential obstacles, and provide basic data for visual perception for the system;

[0012] The distance measurement module is used to measure the distance of obstacles in the environment in real time and determine accurate distance data to adjust the posture and control the movement route of the bionic intelligent fish device in a dynamic environment;

[0013] The environmental perception and stability module is equipped with a lidar and an inertial measurement unit to obtain point cloud data and determine the posture and motion state of the bionic intelligent fish device for environmental mapping and path optimization.

[0014] In some optional embodiments, the visual perception system performs three-dimensional reconstruction of the environment based on the image data, including:

[0015] The data collected by two different image acquisition units in the image data are obtained, and the epipolar geometric relationship of the pixel coordinates in the image data is determined by the following formula:

[0016]

[0017] in, x 1, x 2 is the homogeneous coordinate of the pixel matching point in the images collected by different image acquisition units, F is the basic matrix, describing the epipolar constraint relationship between the images captured by the two image acquisition units, and T represents the transpose;

[0018] The relationship between the depth Z of a pixel and the disparity d is as follows:

[0019]

[0020] Z is the depth of the 3D scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between the two image acquisition units; d is the disparity, which represents the horizontal pixel coordinate difference between the matching points in the left and right views;

[0021] Determine the projection model for the image acquisition unit and project the 3D point X to the image point x based on the projection model:

[0022]

[0023] x is the homogeneous coordinate of the two-dimensional pixel point, K is the camera internal parameter matrix, including the camera location x、y Focal length of two axes fx , fy , principal point coordinates c x , c y ; ;[ R ∣ t ] is the camera extrinsic matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of the 3D scene point;

[0024] Determine the light intensity consistency error of the 3D scene and compensate for the light intensity error of the pixels;

[0025] Determining the loss function for depth estimation of 3D scenes based on neural networks , and use the loss function to correct the pixel parameters respectively:

[0026]

[0027] Among them, L depth is the deep regression loss, L degree is the gradient consistency loss to force the depth map edges to align with the image edges; L smooth It is a smoothness constraint to avoid noise in the depth map; λd 、 λg 、 λs are the weight coefficients of each loss item respectively;

[0028] Render the 3D scene as follows:

[0029]

[0030] For rays r The final rendered color, s i For the i The volume density of the sampling points, c i For the i The color of the sampling point, d i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is:

[0031] .

[0032] In some optional embodiments, the path planning system generates an optimal path from a starting point to a target point for the bionic intelligent fish device based on the three-dimensional reconstructed environment, including:

[0033] For each point p in the point cloud, the nearest centroid point set Select the kth nearest neighbor centroid point and construct the covariance matrix:

[0034]

[0035] in, , is the neighborhood centroid;

[0036] Perform eigenvalue decomposition on the covariance matrix C:

[0037]

[0038] The eigenvector corresponding to the minimum eigenvalue is the normal vector estimate:

[0039] , ,in, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix;

[0040] The point with the smallest curvature is selected as the initial seed, and the normal vector of the point cloud data is determined based on the initial seed. The curvature is calculated as follows:

[0041]

[0042] Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function ,as follows:

[0043]

[0044] in, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth,

[0045] Based on the kernel function, mean shift clustering is performed on the sphere as follows:

[0046]

[0047] is the mean shift vector, is the normalized normal vector of the current point, The first i The normalized normal vector of the point, is the kernel function, n is the total number of points in the neighborhood;

[0048] Perform a cost search in the 3D reconstructed environment map:

[0049] in, g ( n ) is from the starting point to the node n The actual cost, h ( n ) is the heuristic estimated cost;

[0050] Sampling is performed in three-dimensional space in the following way:

[0051]

[0052] in, d is the step length, q rand are random sampling points, q near is the nearest neighbor node;

[0053] The planned path is optimized based on the gradient information of the cost map as follows:

[0054]

[0055] in, α is the learning rate, is the path point in the k+1th iteration, is the path point in the kth iteration, is the gradient of the cost function;

[0056] Perform B-spline curve fitting on the planned path as follows:

[0057]

[0058] C ( u ) is a point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path point, is the degree of the spline curve;

[0059] Then perform dynamic time warping on the planned path as follows:

[0060]

[0061] P , Q are two paths to be compared, and the point sequences are { p 1,..., p m}、{ q 1,..., q n}, π To align the paths, is the distance function between points.

[0062] In some optional embodiments, the path planning system adjusts the path in real time to avoid obstacles, including:

[0063] The path is represented as a pose sequence with time information, and the path shape and velocity distribution are adjusted by the following formula:

[0064]

[0065] Where X=[x1,...,x N ], is the pose sequence, ,T=[t1,...,t N ], is the timestamp sequence, α, β, γ are weight coefficients;

[0066]

[0067] is the pose x k To the obstacle i distance, s To control the cost decay speed.

[0068] According to a second aspect of the present application, a cluster control method based on bionic intelligent fish devices is provided, which establishes data transmission links between bionic intelligent fish devices and between bionic intelligent fish devices and cloud servers, enabling data interaction between bionic intelligent fish devices and between bionic intelligent fish devices and cloud servers to ensure the transmission of perception data, status information, and task instructions; including:

[0069] Obtain image data of the surrounding environment collected by the bionic intelligent fish device, and perform three-dimensional reconstruction of the environment based on the image data; identify obstacles in the environment, and measure the distance between the obstacles in real time to determine the distance between the bionic intelligent fish device and the obstacles to assist the bionic intelligent fish device in moving in a dynamic environment; use the laser radar on the bionic intelligent fish device to generate point cloud data for environmental mapping and path planning optimization, and use the inertial measurement unit on the bionic intelligent fish device to collect the device's posture and motion state to obtain the bionic intelligent fish device's position and posture in real time, so as to adjust the bionic intelligent fish device's position and posture to ensure that the bionic intelligent fish device maintains stable operation in a complex environment;

[0070] Generate an optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment; adjust the path in real time to avoid obstacles and ensure the safe movement of the bionic intelligent fish device;

[0071] A driving instruction is sent to the bionic intelligent fish device, so that the bionic intelligent fish device drives its own tail fin driving system to simulate the swinging pattern of the fish's tail fin, thereby driving the bionic intelligent fish device to move in the water.

[0072] In some optional embodiments, the three-dimensional reconstruction of the environment based on the image data includes:

[0073] The data collected by two different image acquisition units in the image data are obtained, and the epipolar geometric relationship of the pixel coordinates in the image data is determined by the following formula:

[0074]

[0075] in, x 1, x 2 is the homogeneous coordinate of the pixel matching point in the images collected by different image acquisition units, F is the basic matrix, describing the epipolar constraint relationship between the images captured by the two image acquisition units, and T represents the transpose;

[0076] The relationship between the depth Z of a pixel and the disparity d is as follows:

[0077]

[0078] Z is the depth of the 3D scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between the two image acquisition units; d is the disparity, which represents the horizontal pixel coordinate difference between the matching points in the left and right views;

[0079] Determine the projection model for the image acquisition unit and project the 3D point X to the image point x based on the projection model:

[0080]

[0081] x is the homogeneous coordinate of the two-dimensional pixel point, K is the camera internal parameter matrix, including the camera location x、y Focal length of two axes fx , fy , principal point coordinates c x , c y ; ;[ R ∣ t ] is the camera extrinsic matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of the 3D scene point;

[0082] Determine the light intensity consistency error of the 3D scene and compensate for the light intensity error of the pixels;

[0083] Determining the loss function for depth estimation of 3D scenes based on neural networks , and use the loss function to correct the pixel parameters respectively:

[0084]

[0085] Among them, L depth is the deep regression loss, L degree is the gradient consistency loss to force the depth map edges to align with the image edges; L smooth It is a smoothness constraint to avoid noise in the depth map; λd 、 λg 、 λs are the weight coefficients of each loss item respectively;

[0086] Render the 3D scene as follows:

[0087]

[0088] For rays r The final rendered color, s i For the i The volume density of the sampling points, c i For the i The color of the sampling point, d i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is:

[0089] .

[0090] In some optional embodiments, the three-dimensionally reconstructed environment is used to generate an optimal path from a starting point to a target point for the bionic intelligent fish device, including:

[0091] For each point p in the point cloud, the nearest centroid point set Select the kth nearest neighbor centroid point and construct the covariance matrix:

[0092]

[0093] in, , is the neighborhood centroid;

[0094] Perform eigenvalue decomposition on the covariance matrix C:

[0095]

[0096] The eigenvector corresponding to the minimum eigenvalue is the normal vector estimate:

[0097] , ,in, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix;

[0098] The point with the smallest curvature is selected as the initial seed, and the normal vector of the point cloud data is determined based on the initial seed. The curvature is calculated as follows:

[0099]

[0100] Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function ,as follows:

[0101]

[0102] in, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth,

[0103] Based on the kernel function, mean shift clustering is performed on the sphere as follows:

[0104]

[0105] is the mean shift vector, is the normalized normal vector of the current point, The first iThe normalized normal vector of the point, is the kernel function, n is the total number of points in the neighborhood;

[0106] Perform a cost search in the 3D reconstructed environment map:

[0107] in, g ( n ) is from the starting point to the node n The actual cost, h ( n ) is the heuristic estimated cost;

[0108] Sampling is performed in three-dimensional space in the following way:

[0109]

[0110] in, d is the step length, q rand are random sampling points, q near is the nearest neighbor node;

[0111] The planned path is optimized based on the gradient information of the cost map as follows:

[0112]

[0113] in, α is the learning rate, is the path point in the k+1th iteration, is the path point in the kth iteration, is the gradient of the cost function;

[0114] Perform B-spline curve fitting on the planned path as follows:

[0115]

[0116] C ( u ) is a point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path point, is the degree of the spline curve;

[0117] Then perform dynamic time warping on the planned path as follows:

[0118]

[0119] P , Q are two paths to be compared, and the point sequences are {p 1,..., p m}、{ q 1,..., q n}, π To align the paths, is the distance function between points.

[0120] In some optional embodiments, adjusting the path in real time to avoid obstacles includes:

[0121] The path is represented as a pose sequence with time information, and the path shape and velocity distribution are adjusted by the following formula:

[0122]

[0123] Where X=[x1,...,x N ], is the pose sequence, ,T=[t1,...,t N ], is the timestamp sequence, α, β, γ are weight coefficients;

[0124]

[0125] is the pose x k To the obstacle i distance, s To control the cost decay speed.

[0126] According to a third aspect of the present application, an electronic device is provided, including:

[0127] at least one processor; and

[0128] a memory communicatively connected to the at least one processor; wherein,

[0129] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the steps of the cluster control method based on the bionic intelligent fish device described in the present application.

[0130] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the steps of the cluster control method based on the bionic intelligent fish device described in the present application.

[0131] This application presents a cluster control system, method, and storage medium based on a bionic intelligent fish device. Using a bionic fish-like design, this device achieves efficient autonomous reconnaissance and target identification in complex underwater environments. Utilizing multi-source data fusion technology and a path planning algorithm, the bionic intelligent fish device enables precise navigation and obstacle avoidance in unknown environments. Through incremental updates and multi-target allocation, the bionic intelligent fish device can cover a wide area and perform three-dimensional reconstruction of targets, effectively improving the efficiency and accuracy of underwater reconnaissance. This application implements real-time environmental perception and intelligent behavior control. The bionic intelligent fish device acquires high-precision environmental data through a multimodal perception module. Combined with a reinforcement learning algorithm and local obstacle avoidance capabilities, it can dynamically adjust its path, achieving smooth and efficient movement in complex environments. Furthermore, data can be uploaded to the cloud via an IoT module, enabling knowledge sharing and cluster collaboration, ensuring real-time and secure missions. This system not only exhibits extremely low noise levels, making it suitable for monitoring and reconnaissance missions in sensitive waters, but also boasts a high-performance propulsion system for long endurance and high maneuverability, making it suitable for complex environments such as shallows, coral reefs, and estuaries. In addition, bionic intelligent fish equipment can effectively detect water quality, fish distribution and disaster warning, providing strong technical support for marine ecological protection and resource management.

[0132] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0133] The above and other objects, features and advantages of the exemplary embodiments of the present application will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, in which:

[0134] In the drawings, the same or corresponding reference numerals denote the same or corresponding parts.

[0135] Figure 1 A schematic diagram of the structure of a cluster control system based on a bionic intelligent fish device according to an embodiment of the present application is shown;

[0136] Figure 2 A schematic diagram showing a flow chart of a cluster control method based on a bionic intelligent fish device according to an embodiment of the present application is shown;

[0137] Figure 3 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0138] In order to make the purpose, features, and advantages of this application more obvious and easy to understand, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of this application.

[0139] Figure 1 The schematic diagram of the structure of the cluster control system based on the bionic intelligent fish device in the embodiment of the present application is shown as follows: Figure 1 As shown, the swarm control system based on biomimetic intelligent fish devices in this embodiment includes a visual perception system, a path planning system, a biomimetic propulsion system, and an IoT communication system. The visual perception system combines the intelligent algorithm decision-making of the path planning system with the efficient motion capabilities of the biomimetic propulsion system to achieve real-time data transmission and remote control through the IoT communication system. First, the visual perception system collects multidimensional information about the underwater environment. Its core components include a visual sensing module, a distance measurement module, and an environmental perception and stability module. The visual sensing module uses high-precision image acquisition and depth information to achieve three-dimensional reconstruction of the environment, supporting target recognition and obstacle detection. The distance measurement module uses acoustic or laser technology to measure the distance between obstacles and the device in real time, facilitating obstacle avoidance and path planning. Furthermore, it can be combined with lidar and an inertial measurement unit (IMU) to not only generate precise point cloud data for optimized environmental mapping but also provide feedback on the device's posture and motion status, ensuring stable operation in complex environments and improving system reliability and flexibility.

[0140] Based on environmental information provided by the visual perception system, the path planning system uses an algorithmic layer to make intelligent decisions about the bionic fish's movements. The path planning system comprises a global path planning unit and a local obstacle avoidance unit. Using a reinforcement learning algorithm, the path planning system constructs a global map based on perception data and generates an optimal path covering all target areas. The local obstacle avoidance unit adjusts the path in real time. For example, the Move_base framework and the TimeElasticBand algorithm can be used to optimize the bionic intelligent fish's trajectory, enabling dynamic obstacle avoidance and path correction. This combined global and local path planning strategy ensures efficient navigation of the bionic intelligent fish in complex underwater environments.

[0141] Guided by path planning, the bionic propulsion system of the Bionic Intelligent Fish device is based on bionics and simulates the natural swimming motion of fish, achieving efficient and flexible movement. The bionic propulsion system features a streamlined design and is equipped with a high-performance tail fin drive system and a lightweight, high-strength composite tail fin. The streamlined shape reduces underwater resistance, improving swimming efficiency and stealth. The tail fin drive system precisely controls the frequency and angle of oscillation, mimicking the natural propulsion of fish. The composite tail fin reduces overall weight while providing high-strength support, ensuring system durability and stability. These designs enable the Bionic Intelligent Fish device to move rapidly underwater, turn precisely, and ascend and descend smoothly, adapting to diverse mission requirements.

[0142] The entire system enables intelligent data interaction and remote control through an IoT communication system. This IoT communication system comprises a 5G cloud-edge-end integrated communication network and a data encryption and transmission module. The 5G cloud-edge-end integrated communication network leverages 5G technology to create an efficient real-time data transmission link, seamlessly transferring sensory data, status information, and mission instructions between the bionic fish and the cloud. The data encryption and transmission module ensures data security, preventing information leakage or tampering. It also enables users to remotely monitor the bionic fish's operating status and dynamically adjust its behavior strategies.

[0143] The cluster control system based on the bionic intelligent fish device of the embodiment of the present application forms an intelligent bionic fish system with perception, planning, propulsion and communication functions through the synergy of the visual perception system, the path planning system, the bionic propulsion system and the Internet of Things communication system. Among them, the visual perception system provides environmental information input, the path planning system makes task decisions, the bionic propulsion system is responsible for executing movements, and the Internet of Things communication system ensures the smooth flow of data and instructions. The combination of the four enables the intelligent and efficient operation of bionic fish in complex underwater environments. The cluster control system based on the bionic intelligent fish device of the embodiment of the present application can be widely used in military reconnaissance, marine resource exploration, ecological environment monitoring and other fields, providing strong technical support for the intelligent development of underwater operations.

[0144] In the bionic fish visual perception system of the embodiment of the present application, the visual sensing module is used to collect image data of the surrounding environment and realize three-dimensional reconstruction of the environment through high-precision camera technology. Its main purpose is to identify surrounding objects, targets and potential obstacles, providing a basis for visual perception for the system. This module supports the capture of depth information to accurately estimate the distance between the target and the device and navigate in complex environments; the distance measurement module is used to measure the distance to obstacles in the environment in real time and provide accurate distance data to help the device flexibly adjust and move in a dynamic environment; the environmental perception and stability module uses lidar and inertial measurement unit (IMU) to further enhance environmental perception capabilities and ensure the stability of the device. The lidar is responsible for generating accurate point cloud data to support environmental mapping and optimized path planning, while the IMU provides real-time feedback on the device's posture and motion status to ensure stable operation in complex environments.

[0145] In the path planning system of the embodiment of the present application, the global path planning unit uses a reinforcement learning algorithm to generate the optimal path from the starting point to the target point for the bionic fish, ensuring the overall efficiency of the path planning; the local obstacle avoidance unit, based on the Move_base framework and the TimeElasticBand algorithm, adjusts the path in real time to avoid obstacles, ensuring the safety and flexibility of the bionic fish in complex environments.

[0146] Specifically, the visual perception system of the embodiment of the present application performs three-dimensional reconstruction of the environment based on image data, including:

[0147] The data collected by two different image acquisition units in the image data are obtained, and the epipolar geometric relationship of the pixel coordinates in the image data is determined by the following formula:

[0148]

[0149] in, x 1, x 2 is the homogeneous coordinate of the pixel matching point in the images collected by different image acquisition units, F is the basic matrix, describing the epipolar constraint relationship between the images captured by the two image acquisition units, and T represents the transpose;

[0150] The relationship between the depth Z of a pixel and the disparity d is as follows:

[0151]

[0152] Z is the depth of the 3D scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between the two image acquisition units; dis the disparity, which represents the horizontal pixel coordinate difference between the matching points in the left and right views;

[0153] Determine the projection model for the image acquisition unit and project the 3D point X to the image point x based on the projection model:

[0154]

[0155] x is the homogeneous coordinate of the two-dimensional pixel point, K is the camera internal parameter matrix, including the camera location x、y Focal length of two axes fx , fy , principal point coordinates c x , c y ; ;[ R ∣ t ] is the camera extrinsic matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of the 3D scene point;

[0156] Determine the light intensity consistency error of the 3D scene and compensate for the light intensity error of the pixels;

[0157] Determining the loss function for depth estimation of 3D scenes based on neural networks , and use the loss function to correct the pixel parameters respectively:

[0158]

[0159] Among them, L depth is the deep regression loss, L degree is the gradient consistency loss to force the depth map edges to align with the image edges; L smooth It is a smoothness constraint to avoid noise in the depth map; λd 、 λg 、 λs are the weight coefficients of each loss item respectively;

[0160] Render the 3D scene as follows:

[0161]

[0162] For rays r The final rendered color, s i For the i The volume density of the sampling points, c i For the iThe color of the sampling point, d i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is:

[0163] .

[0164] The path planning system of the embodiment of the present application generates an optimal path from a starting point to a target point for a bionic intelligent fish device based on a three-dimensional reconstructed environment, including:

[0165] For each point p in the point cloud, the nearest centroid point set Select the kth nearest neighbor centroid point and construct the covariance matrix:

[0166]

[0167] in, , is the neighborhood centroid;

[0168] Perform eigenvalue decomposition on the covariance matrix C as follows:

[0169]

[0170] The eigenvector corresponding to the minimum eigenvalue is the normal vector estimate, as follows:

[0171] , ,in, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix;

[0172] The point with the smallest curvature is selected as the initial seed, and the normal vector of the point cloud data is determined based on the initial seed. The curvature is calculated as follows:

[0173]

[0174] Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function ,as follows:

[0175]

[0176] in, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth,

[0177] Based on the kernel function, mean shift clustering is performed on the sphere as follows:

[0178]

[0179] is the mean shift vector, is the normalized normal vector of the current point, The first i The normalized normal vector of the point, is the kernel function, n is the total number of points in the neighborhood;

[0180] Perform a cost search in the 3D reconstructed environment map:

[0181] in, g ( n ) is from the starting point to the node n The actual cost, h ( n ) is the heuristic estimated cost;

[0182] Sampling is performed in three-dimensional space in the following way:

[0183]

[0184] in, d is the step length, q rand are random sampling points, q near is the nearest neighbor node;

[0185] The planned path is optimized based on the gradient information of the cost map as follows:

[0186]

[0187] in, α is the learning rate, is the path point in the k+1th iteration, is the path point in the kth iteration, is the gradient of the cost function;

[0188] Perform B-spline curve fitting on the planned path as follows:

[0189]

[0190] C ( u ) is a point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path point, is the degree of the spline curve;

[0191] Then perform dynamic time warping on the planned path as follows:

[0192]

[0193] P , Q are two paths to be compared, and the point sequences are { p 1,..., p m}、{ q 1,..., q n}, π To align the paths, is the distance function between points.

[0194] The path planning system of the embodiment of the present application adjusts the path in real time to avoid obstacles, including:

[0195] The path is represented as a pose sequence with time information, and the path shape and velocity distribution are adjusted by the following formula:

[0196]

[0197] Where X=[x1,...,x N ], is the pose sequence, ,T=[t1,...,t N ], is the timestamp sequence, α, β, γ are weight coefficients;

[0198]

[0199] is the pose x k To the obstacle i distance, s To control the cost decay speed.

[0200] The cluster control system based on bionic intelligent fish devices in the embodiment of the present application integrates a variety of advanced technologies, such as visual perception, path planning, bionic propulsion and Internet of Things communication, aiming to solve the problems of intelligence and efficiency in tasks such as underwater reconnaissance, marine resource exploration, and environmental monitoring, especially autonomous navigation, obstacle avoidance and task execution in dynamic and complex underwater environments.

[0201] The cluster control system based on the bionic intelligent fish device in the embodiment of the present application adopts a two-layer feedback control mechanism, and the collaborative work of the perception layer and the decision-making layer ensures real-time adjustment and path optimization in complex environments. The perception layer provides real-time environmental data through visual perception and various sensor modules, and the decision-making layer makes intelligent decisions based on this data, dynamically optimizing path planning and obstacle avoidance strategies. Through this two-layer feedback mechanism, the bionic fish can respond quickly in a dynamically changing underwater environment, avoiding the problems of insufficient path planning and environmental adaptability in traditional methods. In addition, the cluster control system based on the bionic intelligent fish device in the embodiment of the present application adopts Internet of Things technology, supports real-time data transmission and remote control, which not only enables the bionic fish to perform autonomous reconnaissance, but also performs task scheduling and monitoring through the cloud platform, further improving its application efficiency and flexibility in complex water environments. Through these innovative designs, the present application not only improves the application performance of bionic fish in underwater reconnaissance, environmental monitoring and other fields, but also provides strong technical support for the intelligent execution of complex water tasks.

[0202] Figure 2 FIG. 1 shows a flow chart of a cluster control method based on a bionic intelligent fish device according to an embodiment of the present application. Figure 2 As shown, the cluster control method based on the bionic intelligent fish device in the embodiment of the present application includes the following processing steps:

[0203] Step 201: construct data transmission links between bionic intelligent fish devices and between bionic intelligent fish devices and cloud servers, so as to enable data interaction between bionic intelligent fish devices and between bionic intelligent fish devices and cloud servers to ensure the transmission of perception data, status information and task instructions.

[0204] In the embodiments of this application, data transmission between bionic intelligent fish devices requires the use of a transmission medium such as water. As an implementation method, blue-green light in the 450-550nm band is used as a communication transmission method to achieve data communication between devices and between devices and relay stations. Extremely low frequency (ELF, 3-300Hz) or ultra-low frequency (SLF) electromagnetic waves can also be used for data transmission. Of course, if the communication area is small, wired communication can also be used for data transmission.

[0205] Step 202: Obtain image data of the surrounding environment collected by the bionic intelligent fish device, and perform three-dimensional reconstruction of the environment based on the image data.

[0206] In the embodiment of the present application, data collected by two different image acquisition units in the image data is obtained, and the epipolar geometric relationship of the pixel coordinates in the image data is determined by the following formula:

[0207]

[0208] in, x 1, x 2 is the homogeneous coordinate of the pixel matching point in the images collected by different image acquisition units, F is the basic matrix, describing the epipolar constraint relationship between the images captured by the two image acquisition units, and T represents the transpose;

[0209] The relationship between the depth Z of a pixel and the disparity d is as follows:

[0210]

[0211] Z is the depth of the 3D scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between the two image acquisition units; d is the disparity, which represents the horizontal pixel coordinate difference between the matching points in the left and right views;

[0212] Determine the projection model for the image acquisition unit and project the 3D point X to the image point x based on the projection model:

[0213]

[0214] x is the homogeneous coordinate of the two-dimensional pixel point, K is the camera internal parameter matrix, including the camera location x、y Focal length of two axes fx , fy , principal point coordinates c x , c y ; ;[ R ∣ t ] is the camera extrinsic matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of the 3D scene point;

[0215] Determine the light intensity consistency error of the 3D scene and compensate for the light intensity error of the pixels;

[0216] Determining the loss function for depth estimation of 3D scenes based on neural networks , and use the loss function to correct the pixel parameters respectively:

[0217]

[0218] Among them, L depth is the deep regression loss, L degreeis the gradient consistency loss to force the depth map edges to align with the image edges; L smooth It is a smoothness constraint to avoid noise in the depth map; λd 、 λg 、 λs are the weight coefficients of each loss item respectively;

[0219] Render the 3D scene as follows:

[0220]

[0221] For rays r The final rendered color, s i For the i The volume density of the sampling points, c i For the i The color of the sampling point, d i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is:

[0222] .

[0223] Step 203 , identifying obstacles in the environment, and performing real-time distance measurement on the obstacles to determine the distance between the bionic intelligent fish device and the obstacles, so as to assist the bionic intelligent fish device in moving in a dynamic environment.

[0224] In this embodiment of the present application, active sonar can be used to transmit sound waves and receive echoes, calculating the distance and shape of obstacles based on the time difference and intensity. Alternatively, multi-beam forward-looking sonar (FLS) technology can be used to generate 2D / 3D images of the forward sector to determine the shape of obstacles. In this embodiment of the present application, laser radar (LiDAR) can also be used to transmit blue-green laser pulses and calculate the distance and shape based on the reflected light.

[0225] In step 204, the laser radar on the bionic intelligent fish device is used to generate point cloud data for environmental mapping and path planning optimization. The inertial measurement unit on the bionic intelligent fish device is used to collect the posture and motion state of the device to obtain the position and posture of the bionic intelligent fish device in real time, so as to adjust the position and posture of the bionic intelligent fish device to ensure that the position and posture of the bionic intelligent fish device remain stable in a complex environment.

[0226] In the embodiments of this application, an inertial measurement unit (IMU) is used to estimate the device's attitude (pitch, roll, yaw) and motion state (velocity, position) by measuring acceleration and angular velocity, combined with sensor fusion and filtering algorithms. An IMU typically consists of the following sensors: a three-axis accelerometer: This measures linear acceleration (including gravity). A three-axis gyroscope: This measures angular velocity (device rotation rate) and integrates this angular velocity to obtain attitude angles (pitch, roll, yaw). A three-axis magnetometer: This measures the direction of the Earth's magnetic field to assist in correcting the yaw angle.

[0227] Step 205 : Generate an optimal path from the starting point to the target point for the bionic intelligent fish device based on the three-dimensional reconstructed environment; adjust the path in real time to avoid obstacles and ensure the safe movement of the bionic intelligent fish device.

[0228] The path planning system generates the optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment. Specifically,

[0229] For each point p in the point cloud, the nearest centroid point set Select the kth nearest neighbor centroid point and construct the covariance matrix:

[0230]

[0231] in, , is the neighborhood centroid;

[0232] Perform eigenvalue decomposition on the covariance matrix C:

[0233]

[0234] The eigenvector corresponding to the minimum eigenvalue is the normal vector estimate:

[0235] , ,in, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix;

[0236] The point with the smallest curvature is selected as the initial seed, and the normal vector of the point cloud data is determined based on the initial seed. The curvature is calculated as follows:

[0237]

[0238] Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function ,as follows:

[0239]

[0240] in, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth,

[0241] Based on the kernel function, mean shift clustering is performed on the sphere as follows:

[0242]

[0243] is the mean shift vector, is the normalized normal vector of the current point, The first i The normalized normal vector of the point, is the kernel function, n is the total number of points in the neighborhood;

[0244] Perform a cost search in the 3D reconstructed environment map:

[0245] in, g ( n ) is from the starting point to the node n The actual cost, h ( n ) is the heuristic estimated cost;

[0246] Sampling is performed in three-dimensional space in the following way:

[0247]

[0248] in, d is the step length, q rand are random sampling points, q near is the nearest neighbor node;

[0249] The planned path is optimized based on the gradient information of the cost map as follows:

[0250]

[0251] in, α is the learning rate, is the path point in the k+1th iteration, is the path point in the kth iteration, is the gradient of the cost function;

[0252] Perform B-spline curve fitting on the planned path as follows:

[0253]

[0254] C ( u) is a point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path point, is the degree of the spline curve;

[0255] Then perform dynamic time warping on the planned path as follows:

[0256]

[0257] P , Q are two paths to be compared, and the point sequences are { p 1,..., p m}、{ q 1,..., q n}, π To align the paths, is the distance function between points.

[0258] The path planning system adjusts the path in real time to avoid obstacles, including:

[0259] The path is represented as a pose sequence with time information, and the path shape and velocity distribution are adjusted by the following formula:

[0260]

[0261] Where X=[x1,...,x N ], is the pose sequence, ,T=[t1,...,t N ], is the timestamp sequence, α, β, γ are weight coefficients;

[0262]

[0263] is the pose x k To the obstacle i distance, s To control the cost decay speed.

[0264] Step 206 : Send a driving instruction to the bionic intelligent fish device, so that the bionic intelligent fish device drives its own tail fin driving system to simulate the swinging pattern of the fish's tail fin, thereby driving the bionic intelligent fish device to move in the water.

[0265] Through the aforementioned driving instructions, etc., the server can send them to different bionic intelligent fish devices so that they can move according to the posture and movement speed determined by the server, so as to avoid overlapping motion paths between bionic intelligent fish devices, avoid collisions, and collect hydrological information and other information in different areas.

[0266] According to an embodiment of the present application, the present application also describes an electronic device and a readable storage medium.

[0267] Figure 3 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0268] like Figure 3 As shown, electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of electronic device 800. Computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to bus 804.

[0269] Multiple components in the electronic device 800 are connected to the I / O interface 805, including an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0270] The computing unit 801 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the swarm control method for biomimetic intelligent fish devices. For example, in some embodiments, the swarm control method for biomimetic intelligent fish devices can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the swarm control method for biomimetic intelligent fish devices described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the cluster control method based on the bionic intelligent fish device in any other appropriate manner (for example, by means of firmware).

[0271] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0272] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0273] In the context of this application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or electronic device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or electronic device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0274] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0275] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0276] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0277] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0278] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features being referred to. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one such feature. Throughout the description of this application, "plurality" means two or more, unless otherwise specifically defined.

[0279] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A cluster control system based on bionic intelligent fish equipment, characterized in that: The system includes a visual perception system, a path planning system, a bionic propulsion system and an Internet of Things communication system; wherein: The visual perception system is used to obtain image data of the surrounding environment collected by the bionic intelligent fish device and perform three-dimensional reconstruction of the environment based on the image data; identify obstacles in the environment and measure the distance between the obstacles in real time to determine the distance between the bionic intelligent fish device and the obstacles to assist the bionic intelligent fish device in moving in a dynamic environment; use the laser radar on the bionic intelligent fish device to generate point cloud data for environmental mapping and path planning optimization, and use the inertial measurement unit on the bionic intelligent fish device to collect the device's posture and motion state to obtain the bionic intelligent fish device's position and posture in real time, so as to adjust the bionic intelligent fish device's position and posture to ensure that the bionic intelligent fish device maintains stable operation in a complex environment; A path planning system is used to generate the optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment. The path is adjusted in real time to avoid obstacles and ensure the safe movement of the bionic intelligent fish device. The bionic propulsion system has a streamlined design and is designed to mimic the fish's bionic appearance. It is equipped with a composite tail fin and a tail fin drive system. The tail fin drive system drives the tail fin to simulate the swinging pattern of the fish's tail fin, propelling the bionic intelligent fish device through the water. The IoT communication system is used to build data transmission links between bionic smart fish devices and between bionic smart fish devices and cloud servers, enabling data interaction between bionic smart fish devices and between bionic smart fish devices and cloud servers to ensure the transmission of perception data, status information and task instructions; The visual perception system performs three-dimensional reconstruction of the environment based on image data, including: The data collected by two different image acquisition units in the image data are obtained, and the epipolar geometric relationship of the pixel coordinates in the image data is determined by the following formula: in, x 1, x 2 is the homogeneous coordinate of the pixel matching point in the images collected by different image acquisition units, F is the basic matrix, describing the epipolar constraint relationship between the images captured by the two image acquisition units, and T represents the transpose; The relationship between the depth Z of a pixel and the disparity d is as follows: Z is the depth of the 3D scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between the two image acquisition units; d is the disparity, which represents the horizontal pixel coordinate difference between the matching points in the left and right views; Determine the projection model for the image acquisition unit and project the 3D point X to the image point x based on the projection model: x is the homogeneous coordinate of the two-dimensional pixel point, K is the camera internal parameter matrix, including the camera location x, y Focal length of two axes fx , fy , principal point coordinates c x , c y ; ;[ R ∣ t ] is the camera extrinsic matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of the 3D scene point; Determine the light intensity consistency error of the 3D scene and compensate for the light intensity error of the pixels; Determining the loss function for depth estimation of 3D scenes based on neural networks , and use the loss function to correct the pixel parameters respectively: Among them, L depth is the deep regression loss, L grad is the gradient consistency loss to force the depth map edges to align with the image edges; L smooth It is a smoothness constraint to avoid noise in the depth map; λd 、 λg 、 λs are the weight coefficients of each loss item respectively; Render the 3D scene as follows: For rays r The final rendered color, σ i For the i The volume density of the sampling points, c i For the i The color of the sampling point, δ i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is: 。 2. The cluster control system according to claim 1, characterized in that: The visual perception system includes: The visual sensing module is used to collect image data of the surrounding environment, identify surrounding objects, targets, and potential obstacles, and provide basic data for visual perception for the system; The distance measurement module is used to measure the distance of obstacles in the environment in real time and determine accurate distance data to adjust the posture and control the movement route of the bionic intelligent fish device in a dynamic environment; The environmental perception and stability module is equipped with a lidar and an inertial measurement unit to obtain point cloud data and determine the posture and motion state of the bionic intelligent fish device for environmental mapping and path optimization.

3. The cluster control system according to claim 1, characterized in that: The path planning system generates an optimal path from a starting point to a target point for the bionic intelligent fish device based on a three-dimensional reconstructed environment, including: For each point p in the point cloud, the nearest centroid point set Select the kth nearest neighbor centroid point and construct the covariance matrix: in, , is the neighborhood centroid; Perform eigenvalue decomposition on the covariance matrix C: The eigenvector corresponding to the minimum eigenvalue is the normal vector estimate: , ,in, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix; The point with the smallest curvature is selected as the initial seed, and the normal vector of the point cloud data is determined based on the initial seed. The curvature is calculated as follows: Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function ,as follows: in, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth, Based on the kernel function, mean shift clustering is performed on the sphere as follows: is the mean shift vector, is the normalized normal vector of the current point, The first i The normalized normal vector of the point, is the kernel function, n is the total number of points in the neighborhood; Perform a cost search in the 3D reconstructed environment map: in, g ( n ) is from the starting point to the node n The actual cost, h ( n ) is the heuristic estimated cost; Sampling is performed in three-dimensional space in the following way: in, δ is the step length, q rand are random sampling points, q near is the nearest neighbor node; The planned path is optimized based on the gradient information of the cost map as follows: in, α is the learning rate, is the path point in the k+1th iteration, is the path point in the kth iteration, is the gradient of the cost function; Perform B-spline curve fitting on the planned path as follows: C ( u ) is a point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path point, is the degree of the spline curve; Then perform dynamic time warping on the planned path as follows: P , Q are two paths to be compared, and the point sequences are { p 1,..., p m }、{ q 1,..., q n }, π To align the paths, is the distance function between points.

4. The cluster control system according to claim 1, characterized in that: The path planning system adjusts the path in real time to avoid obstacles, including: The path is represented as a pose sequence with time information, and the path shape and velocity distribution are adjusted by the following formula: Where X=[x1,...,x N ], is the pose sequence, ,T=[t1,...,t N ], is the timestamp sequence, α, β, γ are weight coefficients; is the pose x k To the obstacle i distance, σ To control the cost decay speed.

5. A cluster control method based on bionic intelligent fish equipment, characterized in that: Constructing data transmission links between bionic intelligent fish devices and between bionic intelligent fish devices and cloud servers, enabling data interaction between bionic intelligent fish devices and between bionic intelligent fish devices and cloud servers to ensure the transmission of perception data, status information and task instructions; the method includes: Obtain image data of the surrounding environment collected by the bionic intelligent fish device, and perform three-dimensional reconstruction of the environment based on the image data; identify obstacles in the environment, and measure the distance between the obstacles in real time to determine the distance between the bionic intelligent fish device and the obstacles to assist the bionic intelligent fish device in moving in a dynamic environment; use the laser radar on the bionic intelligent fish device to generate point cloud data for environmental mapping and path planning optimization, and use the inertial measurement unit on the bionic intelligent fish device to collect the device's posture and motion state to obtain the bionic intelligent fish device's position and posture in real time, so as to adjust the bionic intelligent fish device's position and posture to ensure that the bionic intelligent fish device maintains stable operation in a complex environment; Generate an optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment; adjust the path in real time to avoid obstacles and ensure the safe movement of the bionic intelligent fish device; Sending a driving instruction to the bionic intelligent fish device so that the bionic intelligent fish device drives its own tail fin driving system to simulate the swinging pattern of the fish's tail fin, thereby driving the bionic intelligent fish device to move in the water; The three-dimensional reconstruction of the environment based on the image data includes: The data collected by two different image acquisition units in the image data are obtained, and the epipolar geometric relationship of the pixel coordinates in the image data is determined by the following formula: in, x 1, x 2 is the homogeneous coordinate of the pixel matching point in the images collected by different image acquisition units, F is the basic matrix, describing the epipolar constraint relationship between the images captured by the two image acquisition units, and T represents the transpose; The relationship between the depth Z of a pixel and the disparity d is as follows: Z is the depth of the 3D scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between the two image acquisition units; d is the disparity, which represents the horizontal pixel coordinate difference between the matching points in the left and right views; Determine the projection model for the image acquisition unit and project the 3D point X to the image point x based on the projection model: x is the homogeneous coordinate of the two-dimensional pixel point, K is the camera internal parameter matrix, including the camera location x, y Focal length of two axes fx , fy , principal point coordinates c x , c y ; ;[ R ∣ t ] is the camera extrinsic matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of the 3D scene point; Determine the light intensity consistency error of the 3D scene and compensate for the light intensity error of the pixels; Determining the loss function for depth estimation of 3D scenes based on neural networks , and use the loss function to correct the pixel parameters respectively: Among them, L depth is the deep regression loss, L grad is the gradient consistency loss to force the depth map edges to align with the image edges; L smooth It is a smoothness constraint to avoid noise in the depth map; λd 、 λg 、 λs are the weight coefficients of each loss item respectively; Render the 3D scene as follows: For rays r The final rendered color, σ i For the i The volume density of the sampling points, c i For the i The color of the sampling point, δ i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is: 。 6. The cluster control method according to claim 5, characterized in that: The three-dimensional reconstruction-based environment generates an optimal path from a starting point to a target point for the bionic intelligent fish device, including: For each point p in the point cloud, the nearest centroid point set Select the kth nearest neighbor centroid point and construct the covariance matrix: in, , is the neighborhood centroid; Perform eigenvalue decomposition on the covariance matrix C: The eigenvector corresponding to the minimum eigenvalue is the normal vector estimate: , ,in, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix; The point with the smallest curvature is selected as the initial seed, and the normal vector of the point cloud data is determined based on the initial seed. The curvature is calculated as follows: Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function ,as follows: in, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth, Based on the kernel function, mean shift clustering is performed on the sphere as follows: is the mean shift vector, is the normalized normal vector of the current point, The first i The normalized normal vector of the point, is the kernel function, n is the total number of points in the neighborhood; Perform a cost search in the 3D reconstructed environment map: in, g ( n ) is from the starting point to the node n The actual cost, h ( n ) is the heuristic estimated cost; Sampling is performed in three-dimensional space in the following way: in, δ is the step length, q rand are random sampling points, q near is the nearest neighbor node; The planned path is optimized based on the gradient information of the cost map as follows: in, α is the learning rate, is the path point in the k+1th iteration, is the path point in the kth iteration, is the gradient of the cost function; Perform B-spline curve fitting on the planned path as follows: C ( u ) is a point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path point, is the degree of the spline curve; Then perform dynamic time warping on the planned path as follows: P , Q are two paths to be compared, and the point sequences are { p 1,..., p m }、{ q 1,..., q n }, π To align the paths, is the distance function between points.

7. The cluster control method according to claim 5, characterized in that: The real-time adjustment of the path to avoid obstacles includes: The path is represented as a pose sequence with time information, and the path shape and velocity distribution are adjusted by the following formula: Where X=[x1,...,x N ], is the pose sequence, ,T=[t1,...,t N ], is the timestamp sequence, α, β, γ are weight coefficients; is the pose x k To the obstacle i distance, σ To control the cost decay speed.

8. A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the steps of the cluster control method based on the bionic intelligent fish device as described in any one of claims 5 to 7.

Images