Cluster control system and method based on bionic intelligent fish equipment, and storage medium

Abstract

The invention provides a cluster control system and method based on bionic intelligent fish equipment, and a storage medium, and the system comprises a visual perception system which carries out the three-dimensional reconstruction of an environment based on image data; an obstacle in the environment is recognized, real-time distance measurement is conducted on the obstacle, and the distance between the bionic intelligent fish equipment and the obstacle is determined; environment mapping and path planning optimization are carried out based on the point cloud data, and the pose of the bionic intelligent fish equipment is obtained in real time, so that the pose of the bionic intelligent fish equipment is adjusted; the path planning system is used for generating an optimal path from a starting point to a target point for the bionic intelligent fish equipment based on a three-dimensional reconstruction environment; the bionic propulsion system is in a fish bionic shape and is in a streamline design; a tail fin made of a composite material and a tail fin driving system are arranged; the tail fin driving system drives the tail fin to simulate the swinging mode of the fish tail fin, and the bionic intelligent fish equipment is pushed to move. The stable and efficient movement of the bionic intelligent fish equipment in a complex environment is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of bionic intelligent underwater robots, especially a smart bionic fish navigation and decision-making system based on double-layer feedback control. Background Art

[0002] With the rapid development of marine economy and technology, traditional monitoring means can no longer meet the increasingly complex requirements. The environmental complexity of underwater tasks, the high requirements for accuracy, and the need for real-time response capabilities pose many challenges to the existing technologies. As an underwater robot with flexibility and low-noise characteristics, the design and application of bionic fish have become a research hotspot in this field. However, there are many problems with existing bionic fish, such as limited sensing ability, low path planning efficiency, insufficient environmental adaptability, etc. Especially, the autonomy and intelligence levels in complex environments still need to be improved.

[0003] The path planning of existing technologies mostly relies on static algorithms and local obstacle avoidance designs, lacking the ability to handle dynamic obstacles and real-time environmental changes. In addition, the control methods of most bionic fish systems are relatively single, failing to fully utilize their advantages in complex dynamic environments, resulting in low task execution efficiency and poor adaptability. Based on this, a smart bionic fish navigation and decision-making system based on double-layer feedback control is proposed. By integrating visual perception, intelligent path planning, bionic propulsion, and Internet of Things technologies, the intelligence level of bionic fish and the efficiency of underwater operations are significantly improved. Summary of the Invention

[0004] The present application provides a cluster control system, method, and storage medium based on bionic intelligent fish devices to at least solve the above technical problems existing in the prior art.

[0005] According to the first aspect of the present application, a cluster control system based on bionic intelligent fish devices is provided. The system includes a visual perception system, a path planning system, a bionic propulsion system, and an Internet of Things communication system; wherein: The visual perception system is used to obtain image data of the surrounding environment collected by the bionic intelligent fish device, perform three-dimensional reconstruction of the environment based on the image data; identify obstacles in the environment, and perform real-time ranging on the obstacles to determine the distance between the bionic intelligent fish device and the obstacles, so as to assist the bionic intelligent fish device to move in a dynamic environment; generate point cloud data using the lidar on the bionic intelligent fish device for environmental mapping and path planning optimization, and collect the attitude and motion state of the device using the inertial measurement unit on the bionic intelligent fish device to obtain the pose of the bionic intelligent fish device in real time, so as to adjust the pose of the bionic intelligent fish device to ensure the stable operation of the pose of the bionic intelligent fish device in a complex environment; A path planning system for generating an optimal path from a starting point to a target point for a bionic intelligent fish device based on a three-dimensional reconstructed environment; adjusting the path in real time to avoid obstacles and ensure the safe movement of the bionic intelligent fish device; A bionic propulsion system, which has a fish-like bionic shape and is of a streamlined design; is provided with a caudal fin made of composite material and a caudal fin drive system; the caudal fin drive system drives the caudal fin to simulate the swinging mode of a fish's caudal fin and pushes the bionic intelligent fish device to move in water; An Internet of Things communication system for building a data transmission link between bionic intelligent fish devices and between bionic intelligent fish devices and a cloud server, enabling data interaction between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server to ensure the transmission of perception data, status information, and task instructions.

[0006] In some optional embodiments, the visual perception system includes: A visual sensing module for collecting image data of the surrounding environment, identifying surrounding objects, targets, and potential obstacles, and providing basic data for visual perception of the system; A distance measurement module for performing real-time ranging on obstacles in the environment to determine accurate distance data for attitude adjustment and movement route control of the bionic intelligent fish device in a dynamic environment; An environment perception and stability module provided with a lidar and an inertial measurement unit to obtain point cloud data, determine the attitude and movement state of the bionic intelligent fish device, and perform environment mapping and path optimization.

[0007] In some optional embodiments, the visual perception system performs three-dimensional reconstruction of the environment based on image data, including: Obtaining data collected by two different image acquisition units in the image data, and determining the epipolar geometry relationship of pixel point coordinates in the image data through the following formula:

[0008] where, x 1, x 2 are the homogeneous coordinates of pixel matching points in images collected by different image acquisition units, F is the fundamental matrix, describing the epipolar constraint relationship between images collected by two image acquisition units, and T represents the transpose; The relationship between the depth Z and the disparity d of a pixel point is as follows:

[0009] Z is the depth of the three-dimensional scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between two image acquisition units; dThe parallax represents the horizontal pixel coordinate difference of the matching points in the left and right views; The image acquisition unit determines the projection model and projects the 3D point X onto the image point x based on the projection model:

[0010] x is the homogeneous coordinate of the 2D pixel point, K is the camera internal parameter matrix, which includes the focal lengths of the camera along x, y two axes, fx , fy , and the principal point coordinates c x , c y ; ; R ∣ t is the camera external parameter matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of the 3D scene point; Determine the light intensity consistency error of the 3D scene and perform light intensity error compensation on the pixel points; Determine the loss function for depth estimation of the 3D scene based on the neural network , and use the loss function to correct the pixel parameters respectively:

[0011] where L depth is the depth regression loss, and L grad is the gradient consistency loss to force the depth map edges to align with the image edges; L smooth is the smoothness constraint to avoid noise in the depth map; λd , λg , λs are the weight coefficients of each loss term respectively; Render the 3D scene as follows:

[0012] is the ray r The finally rendered color, σ i is the i volume density of the c i is the i color of the δ i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is: 。

[0013] In some alternative embodiments, the path planning system generates an optimal path from a starting point to a target point for the bionic intelligent fish device based on the three-dimensional reconstructed environment, including: For each point p in the point cloud, select the k-th nearest centroid point in the set of nearest centroid points to construct a covariance matrix:

[0014] where, , is the neighborhood centroid; Perform eigenvalue decomposition on the covariance matrix C:

[0015] The eigenvector corresponding to the minimum eigenvalue is the normal vector estimation: , ,where, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix; Select the point with the minimum curvature as the initial seed, and determine the normal vector of the point cloud data based on the initial seed. The curvature calculation method is as follows:

[0016] Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function as follows:

[0017] where, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth, Based on the kernel function, perform mean shift clustering on the sphere as follows:

[0018] is the mean shift vector, is the normalized normal vector of the current point, is the normalized normal vector of the i -th point in the neighborhood, is the kernel function, and n is the total number of points in the neighborhood; Perform cost search in the three-dimensional reconstructed environment map: where, g ( n ) is from the starting point to the noden The actual cost h ( n ) is the heuristic estimated cost; Sampling is performed in three - dimensional space in the following manner:

[0019] where, δ is the step size, q rand is the randomly sampled point, q near is the nearest neighbor node; The planned path is optimized based on the gradient information of the cost map as follows:

[0020] where, α is the learning rate, is the path point in the (k + 1)-th iteration, is the path point in the k - th iteration, is the gradient of the cost function; The planned path is fitted with a B - spline curve as follows:

[0021] C ( u ) is the point on the parameterized curve, is the curve parameter, is the i - th control point, generated from the original path points, is the degree of the spline curve; Then, the dynamic time warping is performed on the planned path as follows:

[0022] P , Q are two paths to be compared, and the point sequences are { p 1,..., p m}, { q 1,..., q n}, π is the aligned path, is the point - to - point distance function.

[0023] In some alternative embodiments, the path planning system adjusts the path in real - time to avoid obstacles, including: The path is represented as a sequence of poses with time information, and the path shape and speed distribution are adjusted by the following formula:

[0024] Among them, X = [x1,..., x N , is a pose sequence, , T = [t1,..., t N , is a timestamp sequence, and α, β, γ are weight coefficients;

[0025] is the distance from the pose x k to the obstacle o i . σ is the control cost attenuation rate.

[0026] According to the second aspect of the present application, a swarm control method based on a bionic intelligent fish device is provided. A data transmission link is constructed between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server, enabling data interaction between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server to ensure the transmission of perception data, status information, and task instructions; including: Obtain image data of the surrounding environment collected by the bionic intelligent fish device, and perform three-dimensional reconstruction of the environment based on the image data; identify obstacles in the environment, and perform real-time ranging on the obstacles to determine the distance between the bionic intelligent fish device and the obstacles to assist the bionic intelligent fish device in moving in a dynamic environment; use the lidar on the bionic intelligent fish device to generate point cloud data for environmental mapping and path planning optimization, and use the inertial measurement unit on the bionic intelligent fish device to collect the attitude and motion state of the device to obtain the pose of the bionic intelligent fish device in real time, so as to adjust the pose of the bionic intelligent fish device to ensure the stable operation of the pose of the bionic intelligent fish device in a complex environment; Generate an optimal path from the starting point to the target point for the bionic intelligent fish device based on the three-dimensionally reconstructed environment; adjust the path in real time to avoid obstacles and ensure the safe progress of the bionic intelligent fish device; Send a driving instruction to the bionic intelligent fish device, so that the bionic intelligent fish device drives its own tail fin drive system to simulate the swinging mode of the fish tail fin and push the bionic intelligent fish device to move in the water.

[0027] In some alternative embodiments, the three-dimensional reconstruction of the environment based on the image data includes: Obtain the data collected by two different image acquisition units in the image data, and the epipolar geometry relationship of the pixel point coordinates in the image data is determined by the following formula:

[0028] Among them, x 1, x2 is the homogeneous coordinate of the pixel matching point in the images collected by different image acquisition units. F is the fundamental matrix, which describes the epipolar constraint relationship between the images collected by two image acquisition units. T represents the transpose. The relationship between the depth Z of the pixel point and the disparity d is as follows:

[0029] Z is the depth of the 3D scene point relative to the image acquisition unit. f is the focal length of the camera. B is the baseline distance between two image acquisition units. d is the disparity, which represents the horizontal pixel coordinate difference of the matching point in the left and right views. is the projection model determined by the image acquisition unit. Based on the projection model, the 3D point X is projected onto the image point x:

[0030] x is the homogeneous coordinate of the 2D pixel point. K is the camera intrinsic matrix, which contains the focal lengths of the camera on the x, y two axes fx , fy , the principal point coordinates c x , c y ; ; R ∣ t is the camera extrinsic matrix. R is the rotation matrix. t is the translation vector. X is the homogeneous coordinate of the 3D scene point. Determine the light intensity consistency error of the 3D scene and perform light intensity error compensation on the pixel points. Determine the loss function of the depth estimation of the 3D scene based on the neural network , and use the loss function to correct the pixel parameters respectively:

[0031] where L depth is the depth regression loss, and L grad is the gradient consistency loss to force the depth map edge to align with the image edge; L smooth is the smoothness constraint to avoid noise in the depth map. λd 、 λg 、 λs are the weight coefficients of each loss term respectively. Render the 3D scene as follows:

[0032] is a ray r The color of the final rendering, σ i is the i volume density of the k-th sampling point, c i is the i color of the k-th sampling point, δ i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is: .

[0033] In some alternative embodiments, the three-dimensional reconstruction-based environment generates an optimal path from a starting point to a target point for the bionic intelligent fish device, including: For each point p in the point cloud, select the k-th nearest centroid point in the set of nearest centroid points to construct a covariance matrix:

[0034] where, , is the neighborhood centroid; Perform eigenvalue decomposition on the covariance matrix C:

[0035] The eigenvector corresponding to the smallest eigenvalue is the normal vector estimate: , , where, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix; Select the point with the minimum curvature as the initial seed, and determine the normal vector of the point cloud data based on the initial seed. The curvature calculation method is as follows:

[0036] Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function , as follows:

[0037] where, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth, Based on the kernel function, perform mean shift clustering on the sphere, as follows:

[0038] is the mean shift vector, is the normalized normal vector of the current point, is the i -th normalized normal vector of the points in the neighborhood, is the kernel function, and n is the total number of points in the neighborhood; Perform cost search in the environmental map for 3D reconstruction: Among them, g ( n ) is the actual cost from the starting point to the node n , h ( n ) is the heuristic estimated cost; Sample in 3D space in the following way:

[0039] Among them, δ is the step size, q rand is the randomly sampled point, q near is the nearest neighbor node; Optimize the planned path based on the gradient information of the cost map as follows:

[0040] Among them, α is the learning rate, is the path point in the (k + 1)-th iteration, is the path point in the k-th iteration, is the gradient of the cost function; Perform B-spline curve fitting on the planned path as follows:

[0041] C ( u ) is the point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path points, is the degree of the spline curve; Then perform dynamic time warping on the planned path as follows:

[0042] P , Q are two paths to be compared, and the point sequences are { p 1,...,p m}, { q 1,..., q n}, π is the alignment path, is the point-to-point distance function.

[0043] In some optional embodiments, the real-time adjustment of the path to avoid obstacles includes: Represent the path as a pose sequence with time information, and adjust the path shape and speed distribution through the following formula:

[0044] where X = [x1,..., x N , is the pose sequence, , T = [t1,..., t N , is the timestamp sequence, and α, β, γ are weight coefficients;

[0045] is the distance from the pose x k to the obstacle o i . σ is the control cost decay rate.

[0046] According to the third aspect of the present application, an electronic device is provided, including: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the steps of the cluster control method based on the bionic intelligent fish device of the present application.

[0047] According to the fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, and the computer instructions are used to cause the computer to execute the steps of the cluster control method based on the bionic intelligent fish device of the present application.

[0048] The cluster control system, method, and storage medium based on bionic intelligent fish devices of the present application achieve efficient autonomous reconnaissance and target recognition in complex underwater environments through bionic fish design. By adopting multi-source data fusion technology and path planning algorithms, precise navigation and obstacle avoidance of bionic intelligent fish devices in unknown environments can be realized. Through incremental update and multi-target allocation technologies, bionic intelligent fish devices can cover a wide area and perform three-dimensional reconstruction of targets, effectively improving the efficiency and accuracy of underwater reconnaissance. The present application realizes real-time environmental perception and intelligent behavior control. Bionic intelligent fish devices obtain high-precision environmental data through multi-modal perception modules, and combined with reinforcement learning algorithms and local obstacle avoidance functions, can dynamically adjust the path and achieve stable and efficient movement in complex environments. At the same time, data can be uploaded to the cloud through the Internet of Things module to achieve knowledge sharing and cluster collaboration, ensuring the real-time nature and security of tasks. In this way, it not only has an extremely low noise level and is suitable for monitoring and reconnaissance tasks in sensitive waters, but can also achieve long-term endurance and high maneuverability through a high-performance propulsion system to adapt to complex areas such as shoals, coral reefs, and estuaries. In addition, bionic intelligent fish devices can effectively detect water quality, fish population distribution, and disaster warnings, providing strong technical support for marine ecological protection and resource management.

[0049] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become easily understood through the following description. Brief Description of the Drawings

[0050] By referring to the detailed description below with reference to the accompanying drawings, the above and other objects, features, and advantages of the exemplary embodiments of the present application will become easily understood. In the drawings, several embodiments of the present application are shown in an exemplary rather than restrictive manner, where: In the drawings, the same or corresponding reference numerals indicate the same or corresponding parts.

[0051] Figure 1 Shows a schematic structural diagram of the cluster control system based on bionic intelligent fish devices according to an embodiment of the present application; Figure 2 Shows a schematic flow diagram of the cluster control method based on bionic intelligent fish devices according to an embodiment of the present application; Figure 3 Shows a schematic composition diagram of an electronic device according to an embodiment of the present application. Detailed Embodiments

[0052] To make the objectives, features, and advantages of this application more obvious and understandable, the following will clearly and completely describe the technical solutions in the embodiments of this application with reference to the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of this application.

[0053] Figure 1 The structural schematic diagram of the swarm control system based on the bionic intelligent fish device in the embodiment of this application is shown. As Figure 1 shown, the swarm control system based on the bionic intelligent fish device in the embodiment of this application includes a visual perception system, a path planning system, a bionic propulsion system, and an Internet of Things communication system. The visual perception system can combine the intelligent algorithm decision-making of the path planning system and the efficient motion ability of the bionic propulsion system to achieve real-time data transmission and remote control through the Internet of Things communication system. First, the visual perception system collects multi-dimensional information of the underwater environment, and its core components include a visual sensing module, a distance measurement module, and an environmental perception and stability module. The visual sensing module realizes the three-dimensional reconstruction of the environment through high-precision image acquisition and depth information acquisition, and supports target recognition and obstacle detection; the distance measurement module measures the distance between the obstacle and the device in real time through acoustic or laser technology to assist in obstacle avoidance and path planning. At the same time, lidar and inertial measurement unit (IMU) can also be combined, which not only generates accurate point cloud data to optimize the environmental mapping, but also provides the attitude and motion state feedback of the device to ensure stable operation in a complex environment and improve the reliability and flexibility of the system.

[0054] Based on the environmental information provided by the visual perception system, the path planning system realizes the intelligent decision-making of the motion of the bionic fish through the algorithm layer. The path planning system includes a global path planning unit and a local obstacle avoidance unit. The path planning system constructs a global map according to the perception data through the reinforcement learning algorithm and generates the optimal path covering all target areas. The local obstacle avoidance unit adjusts the path in real time. As an example, based on the Move_base framework and the TimeElasticBand algorithm, it optimizes the motion trajectory of the bionic intelligent fish device to achieve dynamic obstacle avoidance and path correction. This global and local combined path planning strategy ensures the efficient navigation of the bionic intelligent fish device in a complex underwater environment.

[0055] Under the guidance of path planning, the biomimetic propulsion system of the biomimetic intelligent fish device is based on bionics, simulating the natural swimming of fish to achieve efficient and flexible movement capabilities. The biomimetic propulsion system has a streamlined shape design, and is equipped with a high-performance tail fin drive system and a lightweight and high-strength composite tail fin. The streamlined shape reduces underwater resistance, improves swimming efficiency and concealment; the tail fin drive system simulates the natural propulsion method of fish by precisely controlling the swing frequency and angle; the composite tail fin reduces the overall weight while providing high-strength support to ensure the durability and stability of the system. Through these designs, the biomimetic intelligent fish device can move quickly underwater, turn precisely, float and dive smoothly, and adapt to various task requirements.

[0056] The entire system realizes intelligent data interaction and remote control through the Internet of Things communication system. The Internet of Things communication system includes a 5G cloud-edge-end integrated communication network and a data encryption and transmission module. The 5G cloud-edge-end integrated communication network uses 5G technology to build an efficient real-time data transmission link, seamlessly transmitting sensing data, status information, and task instructions between the biomimetic fish and the cloud; the data encryption and transmission module ensures the security of data, prevents information leakage or tampering, and at the same time supports users to remotely monitor the working status of the biomimetic fish and dynamically adjust the behavior strategy.

[0057] The cluster control system based on the biomimetic intelligent fish device in the embodiments of this application forms a smart biomimetic fish system with sensing, planning, propulsion, and communication functions through the coordinated action of the visual perception system, path planning system, biomimetic propulsion system, and Internet of Things communication system. Among them, the visual perception system provides environmental information input, the path planning system makes task decisions, the biomimetic propulsion system is responsible for executing movements, and the Internet of Things communication system ensures the smooth flow of data and instructions. The four combined achieve intelligent and efficient operation of the biomimetic fish in complex underwater environments. The cluster control system based on the biomimetic intelligent fish device in the embodiments of this application can be widely applied in fields such as military reconnaissance, marine resource exploration, and ecological environment monitoring, providing strong technical support for the intelligent development of underwater operations.

[0058] In the bionic fish vision perception system according to the embodiments of the present application, the vision sensing module is used to collect image data of the surrounding environment and achieve three-dimensional reconstruction of the environment through high-precision imaging technology. Its main purpose is to identify surrounding objects, targets, and potential obstacles, providing a basis for visual perception of the system. This module supports the capture of depth information to accurately estimate the distance between the target and the device and perform navigation in complex environments; the distance measurement module is used to perform real-time ranging on obstacles in the environment and provide accurate distance data to help the device make flexible adjustments and movements in a dynamic environment; the environment perception and stability module uses lidar and an inertial measurement unit (IMU) to further enhance the environment perception ability and ensure the stability of the device. The lidar is responsible for generating accurate point cloud data to support environment mapping and optimize path planning, while the IMU provides real-time feedback on the device's attitude and motion state to ensure stable operation in complex environments.

[0059] In the path planning system according to the embodiments of the present application, the global path planning unit uses a reinforcement learning algorithm to generate an optimal path from the starting point to the target point for the bionic fish, ensuring the overall efficiency of path planning; the local obstacle avoidance unit, based on the Move_base framework and the TimeElasticBand algorithm, adjusts the path in real time to avoid obstacles, ensuring the safety and flexibility of the bionic fish in complex environments.

[0060] Specifically, the vision perception system according to the embodiments of the present application performs three-dimensional reconstruction of the environment based on image data, including: Obtaining data collected by two different image acquisition units in the image data, and the epipolar geometry relationship of pixel point coordinates in the image data is determined by the following formula:

[0061] Wherein, x 1, x 2 are the homogeneous coordinates of the pixel matching points in the images collected by different image acquisition units, F is the fundamental matrix, describing the epipolar constraint relationship between the images collected by the two image acquisition units, and T represents the transpose; The relationship between the depth Z of the pixel point and the disparity d is as follows:

[0062] Z is the depth of the three-dimensional scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between the two image acquisition units; d is the disparity, representing the horizontal pixel coordinate difference of the matching point in the left and right views; Determine a projection model for the image acquisition unit and project the 3D point X onto the image point x based on the projection model:

[0063] x is the homogeneous coordinate of the 2D pixel point, K is the camera internal parameter matrix, which includes the focal lengths of the camera along x, y two axes, fx , fy , the principal point coordinates c x , c y ; ; R | t ] is the camera external parameter matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of the 3D scene point; Determine the light intensity consistency error of the 3D scene and perform light intensity error compensation on the pixel points; Determine the loss function for depth estimation of the 3D scene based on a neural network , and use the loss function to correct the pixel parameters respectively:

[0064] where L depth is the depth regression loss, L grad is the gradient consistency loss to force the depth map edges to align with the image edges; L smooth is the smoothness constraint to avoid noise in the depth map; λd , λg , λs are the weight coefficients of each loss term respectively; Render the 3D scene as follows:

[0065] is the ray r The final rendered color, σ i is the i - th sampling point's volume density, c i is the i - th sampling point's color, δ i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is: .

[0066] The path planning system of the embodiment of the present application generates an optimal path from the starting point to the target point for the bionic intelligent fish device based on the three-dimensional reconstructed environment, including: For each point p in the point cloud, select the k-th nearest centroid point in the set of nearest centroid points to construct a covariance matrix:

[0067] where, , is the neighborhood centroid; Perform eigenvalue decomposition on the covariance matrix C as follows:

[0068] The eigenvector corresponding to the minimum eigenvalue is the normal vector estimation, as follows: , , where, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix; Select the point with the minimum curvature as the initial seed, and determine the normal vector of the point cloud data based on the initial seed. The curvature calculation method is as follows:

[0069] Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function , as follows:

[0070] where, is the normalized normal vector of point p, is the normalized normal vector of point q, h is the kernel function bandwidth, Based on the kernel function, perform mean shift clustering on the sphere, as follows:

[0071] is the mean shift vector, is the normalized normal vector of the current point, is the normalized normal vector of the i -th point in the neighborhood, is the kernel function, and n is the total number of points in the neighborhood; Perform cost search in the three-dimensional reconstructed environment map: where, g ( n ) is the actual cost from the starting point to the node n , h (n ) is the heuristic estimated cost; Sampling is performed in three-dimensional space in the following manner:

[0072] where, δ is the step size, q rand is the randomly sampled point, q near is the nearest neighbor node; The planned path is optimized based on the gradient information of the cost map as follows:

[0073] where, α is the learning rate, is the path point in the (k + 1)-th iteration, is the path point in the k-th iteration, is the gradient of the cost function; The planned path is fitted with a B-spline curve as follows:

[0074] C ( u ) is the point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path points, is the degree of the spline curve; Then, dynamic time warping is performed on the planned path as follows:

[0075] P , Q are two paths to be compared, and the point sequences are { p 1,..., p m}, { q 1,..., q n}, π is the aligned path, is the point-to-point distance function.

[0076] The path planning system of the embodiments of the present application adjusts the path in real time to avoid obstacles, including: Representing the path as a pose sequence with time information, and adjusting the path shape and speed distribution through the following formula:

[0077] where, X = [x1,..., xN , is a pose sequence, , T = [t1,..., t N , is a timestamp sequence, and α, β, γ are weight coefficients;

[0078] is the pose x k to the obstacle o i 's distance, σ is the control cost attenuation rate.

[0079] The cluster control system based on bionic intelligent fish devices in the embodiments of the present application integrates a variety of advanced technologies, such as visual perception, path planning, bionic propulsion, and Internet of Things communication, aiming to solve the problems of intelligence and efficiency in tasks such as underwater reconnaissance, marine resource exploration, and environmental monitoring, especially autonomous navigation, obstacle avoidance, and task execution in dynamic and complex underwater environments.

[0080] The cluster control system based on bionic intelligent fish devices in the embodiments of the present application adopts a double-layer feedback control mechanism. The collaborative work of the perception layer and the decision layer ensures real-time adjustment and path optimization in complex environments. The perception layer provides real-time environmental data through visual perception and various sensor modules, and the decision layer makes intelligent decisions based on these data, dynamically optimizing path planning and obstacle avoidance strategies. Through this double-layer feedback mechanism, the bionic fish can quickly respond in a dynamically changing underwater environment, avoiding the problems of insufficient path planning and environmental adaptability in traditional methods. In addition, the cluster control system based on bionic intelligent fish devices in the embodiments of the present application adopts Internet of Things technology to support real-time data transmission and remote control, which not only enables the bionic fish to conduct autonomous reconnaissance, but also can perform task scheduling and monitoring through the cloud platform, further improving its application efficiency and flexibility in complex water environments. Through these innovative designs, the present application not only improves the application performance of bionic fish in the fields of underwater reconnaissance, environmental monitoring, etc., but also provides strong technical support for the intelligent execution of complex water tasks.

[0081] Figure 2 shows the schematic flow chart of the cluster control method based on bionic intelligent fish devices in the embodiments of the present application. As Figure 2 shown, the cluster control method based on bionic intelligent fish devices in the embodiments of the present application includes the following processing steps: Step 201, construct a data transmission link between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server, so that data interaction can be carried out between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server to ensure the transmission of perception data, status information, and task instructions.

[0082] In the embodiments of the present application, data transmission between bionic intelligent fish devices needs to be carried out with the help of transmission media such as water. As an implementation method, blue-green light in the 450-550 nm band is used as a means of communication transmission to achieve data communication between devices and between devices and relay stations. Data transmission can also be carried out using extremely low frequency (ELF, 3-300 Hz) or super low frequency (SLF) electromagnetic waves. Of course, if the communication area is small, wired communication can also be used for data transmission.

[0083] Step 202: Obtain the image data of the surrounding environment collected by the bionic intelligent fish device, and perform three-dimensional reconstruction on the environment based on the image data.

[0084] In the embodiments of the present application, the data collected by two different image acquisition units in the image data are obtained, and the epipolar geometric relationship of the pixel point coordinates in the image data is determined by the following formula:

[0085] where, x 1, x 2 are the homogeneous coordinates of the pixel matching points in the images collected by different image acquisition units, F is the fundamental matrix, which describes the epipolar constraint relationship between the images collected by two image acquisition units, and T represents the transpose; The relationship between the depth Z of the pixel point and the disparity d is as follows:

[0086] Z is the depth of the three-dimensional scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between two image acquisition units; d is the disparity, which represents the horizontal pixel coordinate difference of the matching point in the left and right views; is the projection model determined by the image acquisition unit, and the three-dimensional point X is projected onto the image point x based on the projection model:

[0087] x is the homogeneous coordinate of the two-dimensional pixel point, K is the camera internal parameter matrix, which includes the camera located at x, y the focal lengths of the two axes fx , fy , the principal point coordinates c x , c y ; ; R ∣ t is the camera external parameter matrix, R is the rotation matrix,t is the translation vector; X is the homogeneous coordinate of the 3D scene point; Determine the light intensity consistency error of the 3D scene, and perform light intensity error compensation on the pixel points; Determine the loss function of the depth estimation of the 3D scene based on the neural network , and use the loss function to correct the pixel parameters respectively:

[0088] where L depth is the depth regression loss, and L grad is the gradient consistency loss to force the depth map edge to align with the image edge; L smooth is the smoothness constraint to avoid noise in the depth map; λd , λg , λs are the weight coefficients of each loss term respectively; Render the 3D scene as follows:

[0089] is the ray r The finally rendered color, σ i is the i th sampling point's volume density, c i is the i th sampling point's color, δ i is the distance between adjacent sampling points, T i is the transmittance, and the calculation formula is: .

[0090] Step 203: Identify the obstacles in the environment, perform real-time ranging on the obstacles, and determine the distance between the bionic intelligent fish device and the obstacles to assist the bionic intelligent fish device in moving in a dynamic environment.

[0091] In the embodiments of the present application, an active sonar can be used to emit sound waves and receive echoes, and calculate the distance and shape of the obstacles through the time difference and intensity. Alternatively, the multi-beam forward-looking sonar (FLS) technology can also be used to generate 2D / 3D images of the front fan-shaped area to determine the shape of the obstacles. The embodiments of the present application can also use a lidar (LiDAR) to emit blue-green laser pulses and calculate the distance and shape through the reflected light.

[0092] Step 204: Generate point cloud data using the lidar on the bionic intelligent fish device for environmental mapping and path planning optimization. Collect the attitude and motion state of the device using the inertial measurement unit on the bionic intelligent fish device to obtain the pose of the bionic intelligent fish device in real time, and adjust the pose of the bionic intelligent fish device to ensure its stable operation in a complex environment.

[0093] In the embodiments of the present application, an inertial measurement unit (IMU) is used to estimate the attitude (pitch, roll, yaw) and motion state (speed, position) of the device by measuring acceleration and angular velocity and combining sensor fusion and filtering algorithms. The IMU usually consists of the following sensors: Triaxial accelerometer: Measures linear acceleration (including gravity). Triaxial gyroscope: Measures angular velocity (device rotation rate), and integrates the angular velocity to obtain the attitude angles (pitch, roll, yaw). Triaxial magnetometer: Measures the direction of the geomagnetic field to assist in correcting the yaw angle.

[0094] Step 205: Generate the optimal path from the starting point to the target point for the bionic intelligent fish device based on the three-dimensional reconstructed environment; adjust the path in real time to avoid obstacles and ensure the safe progress of the bionic intelligent fish device.

[0095] The path planning system generates the optimal path from the starting point to the target point for the bionic intelligent fish device based on the three-dimensional reconstructed environment. Specifically, For each point p in the point cloud, select the k-th nearest centroid point in the set of nearest centroid points to construct the covariance matrix:

[0096] where, , is the neighborhood centroid; Perform eigenvalue decomposition on the covariance matrix C:

[0097] The eigenvector corresponding to the smallest eigenvalue is the normal vector estimate: , where, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix; Select the point with the minimum curvature as the initial seed, and determine the normal vector of the point cloud data based on the initial seed. The curvature calculation method is as follows:

[0098] Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function , as follows:

[0099] Among them, is the normalized normal vector of point p, is the normalized normal vector of point q, and h is the kernel function bandwidth. Based on the kernel function, mean shift clustering is performed on the spherical surface as follows:

[0100] is the mean shift vector, is the normalized normal vector of the current point, is the i th normalized normal vector of the point in the neighborhood, is the kernel function, and n is the total number of points in the neighborhood; Cost search is performed in the environmental map of 3D reconstruction: Among them, g ( n ) is the actual cost from the starting point to node n , h ( n ) is the heuristic estimated cost; Sampling is performed in 3D space in the following way:

[0101] Among them, δ is the step size, q rand is the randomly sampled point, q near is the nearest neighbor node; The planned path is optimized based on the gradient information of the cost map as follows:

[0102] Among them, α is the learning rate, is the path point in the (k + 1)-th iteration, is the path point in the k-th iteration, is the gradient of the cost function; The planned path is fitted with a B-spline curve as follows:

[0103] C ( u ) is the point on the parameterized curve, is the curve parameter, is the i-th control point, generated from the original path points, is the degree of the spline curve; Perform dynamic time warping on the planned path as follows:

[0104] P , Q Let and be two paths to be compared, with point sequences { p 1,..., p m} and { q 1,..., q n} respectively, π be the aligned path, be the function of the distance between points.

[0105] The path planning system adjusts the path in real time to avoid obstacles, specifically including: Represent the path as a pose sequence with time information, and adjust the path shape and speed distribution through the following formula:

[0106] where X = [x1,...,x N is the pose sequence, , T = [t1,...,t N is the time stamp sequence, and α, β, γ are weight coefficients;

[0107] is the distance from the pose x k to the obstacle o i , σ is the control cost attenuation rate.

[0108] Step 206: Send a driving instruction to the bionic intelligent fish device, so that the bionic intelligent fish device drives its own tail fin driving system to simulate the swinging mode of the fish tail fin, and pushes the bionic intelligent fish device to move in the water.

[0109] Through the foregoing driving instructions, etc., they can be sent to different bionic intelligent fish devices through the server, so that they move according to the poses and moving speeds determined by the server, so as to avoid the overlap of the movement paths between the bionic intelligent fish devices, avoid collisions, and collect hydrological and other information in different areas.

[0110] According to the embodiments of the present application, the present application also records an electronic device and a readable storage medium.

[0111] Figure 3FIG. shows a schematic block diagram of an exemplary electronic device 800 that can be used to implement embodiments of the present application. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0112] As Figure 3 shown, the electronic device 800 includes a computing unit 801 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the electronic device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0113] A plurality of components in the electronic device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the electronic device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0114] The computing unit 801 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 executes the various methods and processes described above, such as the swarm control method based on the bionic intelligent fish device. For example, in some embodiments, the swarm control method based on the bionic intelligent fish device can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the swarm control method based on the bionic intelligent fish device described above can be executed. Alternatively, in other embodiments, the computing unit 801 can be configured to execute the swarm control method based on the bionic intelligent fish device by any other suitable means (e.g., by means of firmware).

[0115] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), system-on-a-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0116] The program code for implementing the methods of this application can be written in any combination of one or more programming languages. These program codes can be provided to the processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowchart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as an independent software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0117] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or electronic device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or electronic devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0118] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0119] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0120] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.

[0121] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved, and no limitation is imposed herein.

[0122] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of this application, "a plurality" means two or more, unless otherwise specifically defined.

[0123] As described above, this is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed in this application can easily think of changes or substitutions, which should all be covered by the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claims.

Claims

1. A cluster control system based on a bionic intelligent fish device, characterized in that The system includes a visual perception system, a path planning system, a bionic propulsion system, and an Internet of Things communication system; among which: The visual perception system is used to obtain the image data of the surrounding environment collected by the bionic intelligent fish device, perform 3D reconstruction of the environment based on the image data; identify obstacles in the environment, and perform real-time ranging on the obstacles to determine the distance between the bionic intelligent fish device and the obstacles, so as to assist the bionic intelligent fish device to move in a dynamic environment; generate point cloud data using the lidar on the bionic intelligent fish device for environment mapping and path planning optimization, and collect the attitude and motion state of the device using the inertial measurement unit on the bionic intelligent fish device to obtain the pose of the bionic intelligent fish device in real time, so as to adjust the pose of the bionic intelligent fish device to ensure the stable operation of the bionic intelligent fish device in a complex environment; The path planning system is used to generate the optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment; adjust the path in real time to avoid obstacles and ensure the safe progress of the bionic intelligent fish device; The bionic propulsion system has a fish-like bionic shape and is a streamline design; it is provided with a composite tail fin and a tail fin drive system; the tail fin drive system drives the tail fin to simulate the swing mode of the fish tail fin to push the bionic intelligent fish device to move in the water; The Internet of Things communication system is used to build a data transmission link between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server, enabling data interaction between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server to ensure the transmission of perception data, status information, and task instructions.

2. The cluster control system according to claim 1, characterized in that The visual perception system includes: The visual sensing module is used to collect the image data of the surrounding environment, identify surrounding objects, targets, and potential obstacles, and provide the basic data for visual perception of the system; The distance measurement module is used to perform real-time ranging on obstacles in the environment to determine accurate distance data for attitude adjustment and movement route control of the bionic intelligent fish device in a dynamic environment; The environment perception and stability module is provided with a lidar and an inertial measurement unit to obtain point cloud data, determine the attitude and motion state of the bionic intelligent fish device for environment mapping and path optimization.

3. The cluster control system according to claim 1 or 2, characterized in that, The visual perception system performs 3D reconstruction of the environment based on the image data, including: Obtain the data collected by two different image acquisition units in the image data, and the epipolar geometric relationship of the pixel point coordinates in the image data is determined by the following formula: Among them, x 1, x 2 are the homogeneous coordinates of the pixel matching points in the images collected by different image acquisition units, F is the fundamental matrix, describing the epipolar constraint relationship between the images collected by two image acquisition units, and T represents the transpose; The relationship between the depth Z of the pixel point and the disparity d is as follows: Z is the depth of the three-dimensional scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between two image acquisition units; d is the disparity, representing the horizontal pixel coordinate difference of the matching points in the left and right views; Determine the projection model for the image acquisition unit, and project the 3D point X onto the image point x based on the projection model: x is the homogeneous coordinate of a two-dimensional pixel point, K is the camera internal parameter matrix, which contains the camera located at x, y the focal lengths of the two axes fx , fy , and the principal point coordinates c x , c y ; ; R | t is the camera external parameter matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of a three-dimensional scene point; Determine the light intensity consistency error of the 3D scene and perform light intensity error compensation on the pixel points; Loss function for depth estimation of a three-dimensional scene determined based on a neural network and use the loss function to correct the pixel parameters respectively: Among them, L depth is the depth regression loss, and L grad is the gradient consistency loss to enforce the alignment of the depth map edges with the image edges; L smooth is the smoothness constraint to avoid noise in the depth map; λd , λg , λs are the weight coefficients of each loss term respectively; Render the 3D scene as follows: is a ray r The finally rendered color σ i is the i volume density of the nth sampling point c i is the i color of the nth sampling point δ i is the distance between adjacent sampling points T i is the transmittance, and the calculation formula is: 。 4. The cluster control system according to claim 3, characterized in that, The path planning system generates the optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment, including: For each point p in the point cloud, select the k-th nearest centroid point from the set of nearest centroid points to construct a covariance matrix: Among them, , is the neighborhood centroid; Perform eigenvalue decomposition on the covariance matrix C: The eigenvector corresponding to the minimum eigenvalue is the normal vector estimation: , , where is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix; Select the point with the minimum curvature as the initial seed, and determine the normal vector of the point cloud data based on the initial seed. The curvature calculation method is as follows: Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function , as follows: Among them, is the normalized normal vector of point p, is the normalized normal vector of point q, and h is the kernel function bandwidth. Perform mean shift clustering on the spherical surface based on the kernel function, as follows: is the mean shift vector, is the normalized normal vector of the current point, is the i -th normalized normal vector of points within the neighborhood, is the kernel function, and n is the total number of points within the neighborhood; Perform cost search in the environmental map of 3D reconstruction: Among them, g ( n ) is the actual cost from the starting point to the node n , h ( n ) is the heuristic estimated cost; Sample in the 3D space in the following way, as follows: Among them, δ is the step size, q rand is the random sampling point, q near is the nearest neighbor node; Optimize the planned path based on the gradient information of the cost map, as follows: Among them, α is the learning rate, is the path point in the (k + 1)-th iteration, is the path point in the k-th iteration, is the gradient of the cost function; Fit the planned path with a B-spline curve, as follows: C ( u ) is a point on the parametric curve, is the curve parameter, is the i-th control point, generated from the original path points, is the degree of the spline curve; Then perform dynamic time warping on the planned path, as follows: P , Q are two paths to be compared, with the point sequences being { p 1,..., p m} and { q 1,..., q n}, π is the alignment path, is the point - to - point distance function.

5. The cluster control system according to claim 3, wherein The path planning system adjusts the path in real time to avoid obstacles, including: Represent the path as a pose sequence with time information, and adjust the path shape and speed distribution through the following formula: Among them, X = [x1,..., x N , is a pose sequence, , T = [t1,..., t N , is a timestamp sequence, and α, β, γ are weight coefficients; is the pose x k to the obstacle o i distance, σ is the control cost attenuation rate.

6. A cluster control method based on a bionic intelligent fish device, characterized in that, Construct a data transmission link between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server, enabling data interaction between bionic intelligent fish devices and between bionic intelligent fish devices and the cloud server to ensure the transmission of perception data, status information, and task instructions; The method includes: Obtain the image data of the surrounding environment collected by the bionic intelligent fish device, and perform 3D reconstruction of the environment based on the image data; Identify obstacles in the environment, and perform real-time ranging on the obstacles to determine the distance between the bionic intelligent fish device and the obstacles, so as to assist the bionic intelligent fish device in moving in a dynamic environment; Use the lidar on the bionic intelligent fish device to generate point cloud data for environmental mapping and path planning optimization, and use the inertial measurement unit on the bionic intelligent fish device to collect the attitude and motion state of the device to obtain the pose of the bionic intelligent fish device in real time, so as to adjust the pose of the bionic intelligent fish device to ensure the stable operation of the bionic intelligent fish device in a complex environment; Generate the optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment; Adjust the path in real time to avoid obstacles and ensure the safe progress of the bionic intelligent fish device; Send a driving instruction to the bionic intelligent fish device, so that the bionic intelligent fish device drives its own tail fin drive system to simulate the swing mode of the fish tail fin and push the bionic intelligent fish device to move in the water.

7. The cluster control method according to claim 6, wherein The 3D reconstruction of the environment based on the image data includes: Obtain the data collected by two different image acquisition units in the image data, and determine the epipolar geometric relationship of the pixel point coordinates in the image data through the following formula: Among them, x 1, x 2 are the homogeneous coordinates of the pixel matching points in the images collected by different image acquisition units, F is the fundamental matrix, which describes the epipolar constraint relationship between the images collected by two image acquisition units, and T represents the transpose; The relationship between the depth Z of the pixel point and the disparity d is as follows: Z is the depth of the three-dimensional scene point relative to the image acquisition unit; f is the focal length of the camera; B is the baseline distance between two image acquisition units; d is the parallax, representing the horizontal pixel coordinate difference of the matching points in the left and right views; Determine the projection model for the image acquisition unit, and project the 3D point X onto the image point x based on the projection model: x is the homogeneous coordinate of a two-dimensional pixel point, K is the camera intrinsic matrix, which contains the camera's focal lengths along x, y two axes fx , fy , and the coordinates of the principal point c x , c y ; ; R | t is the camera extrinsic matrix, R is the rotation matrix, t is the translation vector; X is the homogeneous coordinate of a three-dimensional scene point; Determine the light intensity consistency error of the 3D scene and perform light intensity error compensation on the pixel points; Loss function for depth estimation of a three-dimensional scene determined based on a neural network and use the loss function to correct the pixel parameters respectively: Among them, L depth is the depth regression loss, and L grad is the gradient consistency loss to enforce the alignment of the depth map edges with the image edges; L smooth is the smoothness constraint to avoid noise in the depth map; λd , λg , λs are the weight coefficients of each loss term respectively; Render the 3D scene, as follows: is a ray r The finally rendered color σ i is the i volume density of the nth sampling point c i is the i color of the nth sampling point δ i is the distance between adjacent sampling points T i is the transmittance, and the calculation formula is: 。 8. The cluster control method according to claim 6, wherein The generation of the optimal path from the starting point to the target point for the bionic intelligent fish device based on the 3D reconstructed environment includes: For each point p in the point cloud, select the k-th nearest centroid point from the set of nearest centroid points to construct a covariance matrix: Among them, , is the neighborhood centroid; Perform eigenvalue decomposition on the covariance matrix C: The eigenvector corresponding to the minimum eigenvalue is the normal vector estimation: , , where, is the eigenvector of the covariance matrix, is the eigenvalue of the covariance matrix; Select the point with the minimum curvature as the initial seed, and determine the normal vector of the point cloud data based on the initial seed. The curvature calculation method is as follows: Map the normal vector to the Gaussian sphere and determine the Gaussian kernel function , as follows: Among them, is the normalized normal vector of point p, is the normalized normal vector of point q, and h is the kernel function bandwidth. Perform mean shift clustering on the spherical surface based on the kernel function, as follows: is the mean shift vector, is the normalized normal vector of the current point, is the i -th normalized normal vector of the points within the neighborhood, is the kernel function, and n is the total number of points within the neighborhood; Performing cost search in the environmental map of 3D reconstruction: Among them, g ( n ) is the actual cost from the starting point to the node n , and h ( n ) is the heuristic estimated cost; Sampling in 3D space in the following manner: Among them, δ is the step size, q rand is the randomly sampled point, q near is the nearest neighbor node; Optimizing the planned path based on the gradient information of the cost map as follows: Among them, α is the learning rate, is the path point in the (k + 1)-th iteration, is the path point in the k-th iteration, is the gradient of the cost function; Performing B-spline curve fitting on the planned path as follows: C ( u ) is a point on the parametric curve, is the curve parameter, is the i-th control point, generated from the original path points, is the degree of the spline curve; Then performing dynamic time warping on the planned path as follows: P , Q are two paths to be compared, and the point sequences are { p 1,..., p m}, { q 1,..., q n}, π is the alignment path, is the point-to-point distance function.

9. The cluster control method according to claim 6, wherein The real-time adjustment of the path to avoid obstacles includes: Representing the path as a pose sequence with time information and adjusting the path shape and speed distribution by the following formula: Among them, X = [x1,..., x N , is a pose sequence, , T = [t1,..., t N , is a timestamp sequence, and α, β, γ are weight coefficients; is the pose x k to the obstacle o i distance, σ is the control cost attenuation rate.

10. A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of an electronic device, enable the electronic device to execute the steps of the swarm control method based on a bionic intelligent fish device according to any one of claims 6 to 9.

Images