Unmanned ship autonomous harbor returning system and method integrating BEV technology and A* optimization algorithm

Abstract

The invention comprises two aspects: the first aspect is an unmanned ship autonomous harbor returning system fusing a BEV technology and an optimization A * algorithm, and the system comprises a feature extraction module which is used for extracting multi-view image features and IMU inertial data; the camera and IMU calibration module is used for completing internal and external parameter calibration of each camera, external parameter calibration among multiple cameras and external parameter calibration between an IMU and the camera; the IMU branch extension module comprises an IMU data preprocessing unit, an IMU state vector construction unit, an IMU time synchronization unit and a multi-modal data fusion unit; the BEV environment modeling module is used for processing the multi-view image features fused with the IMU data and generating a BEV feature map; the grid map generation module is used for dividing the BEV feature map into grid maps; the path planning module is used for performing path planning according to the grid map by adopting an improved A * algorithm based on a dynamic Voronoi field; the second method is an unmanned ship autonomous harbor returning method based on the system and fusing the BEV technology and the optimization A * algorithm. The method has the advantages that the sensing precision and navigation safety of the unmanned ship are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of autonomous navigation of unmanned boats, and in particular to an autonomous return-to-port system and method for unmanned boats integrating BEV technology and optimized A* algorithm. Background Art

[0002] With the rapid development of marine resource development and unmanned systems technology, unmanned vehicles (UAVs) are increasingly being used for tasks such as marine environmental monitoring, resource exploration, and offshore patrols. However, the efficient and safe autonomous return to port after completing their missions still faces significant technical challenges. Existing environmental perception technologies often rely on the fusion of vision and lidar. While this improves perception accuracy, the high cost of lidar limits its widespread application. Furthermore, pure vision solutions, in surface environments, are susceptible to continuous changes in the camera's perspective due to wave-induced pitch motion, making it difficult to stably extract multi-view image features. Existing methods fail to fully integrate motion data such as angular velocity and acceleration from the inertial measurement unit (IMU), leading to the accumulation of image feature alignment errors and affecting the spatiotemporal consistency of BEV feature modeling. Furthermore, incomplete temporal synchronization, coordinate transformation, and feature-level fusion mechanisms between IMU and visual data further limit the accuracy of dynamic obstacle detection.

[0003] Traditional BEV generation methods rely on spatial feature extraction from single-frame images and lack effective utilization of historical temporal information, resulting in deviations in the prediction of dynamic obstacle trajectories. Furthermore, the feature distortion caused by the motion of the unmanned boat is not optimized through the temporal encoder, resulting in insufficient robustness in BEV modeling in dynamic scenarios and difficulty adapting to real-time navigation requirements in complex water surface environments. Furthermore, the A* algorithm, widely used in the field of path planning, uses path length as the optimization objective and does not quantify the threat level of dynamic obstacles, resulting in planned paths that are too close to obstacles and difficult to adapt to the safety requirements of scenarios such as multi-ship flows and complex docks. Although reinforcement learning methods have been attempted for path optimization, their high computational cost and long training cycles limit their practical application. Furthermore, existing technologies lack a risk assessment mechanism based on dynamic Voronoi fields and are unable to integrate the impact of obstacle speed, distance, and motion trends on path safety in real time, resulting in insufficient safety in global path planning.

[0004] In summary, existing technologies suffer from core flaws such as imperfect multimodal data fusion mechanisms, poor spatiotemporal consistency of BEV features, and a lack of dynamic risk assessment for path planning, which severely restrict the reliability and safety of autonomous return-to-port unmanned vehicles. To address these issues, this paper proposes an autonomous return-to-port system for unmanned vehicles that integrates BEV technology with an optimized A* algorithm. This system achieves efficient multimodal data fusion by expanding the IMU branch and optimizing path planning using a dynamic Voronoi field risk function. This significantly improves perception accuracy and navigation safety in complex water environments. Summary of the Invention

[0005] The purpose of the present invention is to overcome the above-mentioned defects in the prior art and provide an unmanned boat autonomous return to port system and method that integrates BEV technology and optimized A* algorithm. The efficient fusion of multimodal data is achieved through the IMU branch expansion module, and a dynamic Voronoi field risk function is designed to optimize path planning, thereby significantly improving the perception accuracy and navigation safety in complex water surface environments, so that the unmanned boat autonomous return to port system can perceive the marine environment in real time, plan the return path, and improve the unmanned boat's environmental perception accuracy and navigation safety.

[0006] To achieve the above object, the present invention is implemented through the following two aspects:

[0007] First, a system for autonomous return to port of unmanned boats that integrates BEV technology and optimized A* algorithm is provided, which includes:

[0008] The feature extraction module is used to capture multi-view images from multiple cameras of the unmanned boat, extract multi-view image features through a neural network model, and collect multiple different IMU inertial data from the unmanned boat's inertial measurement unit (IMU);

[0009] The camera and IMU calibration module is used to complete the internal and external parameter calibration of each camera, the external parameter calibration between multiple cameras, and the external parameter calibration between the IMU and the camera;

[0010] The IMU branch expansion module includes an IMU data preprocessing unit, an IMU state vector construction unit, an IMU time synchronization unit, and a multimodal data fusion unit. The multimodal data fusion unit is used to map the collected IMU data to the multi-view image feature space through the coordinate system transformation relationship, and then fuse the IMU data with the feature maps of each view image;

[0011] A BEV environment modeling module is used to input multi-view image features and fused IMU data into the BEV environment modeling module to generate a BEV feature map representing the unmanned boat environment;

[0012] A grid map generation module is used to divide the BEV feature map generated by the BEV environment modeling module into grid maps of passable areas and obstacle areas;

[0013] The path planning module uses the improved A* algorithm based on the Voronoi field to plan the path for the unmanned boat to return to the port based on the grid map.

[0014] Preferably, the camera and IMU calibration module is used to perform the following process:

[0015] In the calibration of a single camera, Zhang Zhengyou calibration method is used to obtain 3D spatial points and 2D pixel coordinates through a checkerboard calibration plate, identify corner points, establish homography constraints, solve internal and external parameter matrices, and realize the conversion from image coordinate system to world coordinate system.

[0016] In the external parameter calibration between multiple cameras, a checkerboard calibration plate is fixed in the common viewing area of ​​the unmanned boat, and multiple cameras are triggered to collect calibration images at the same time. The initial transformation matrix of each camera coordinate system is solved by the perspective n-point algorithm; the angular velocity and acceleration data of the IMU and the image stream of multiple cameras are collected synchronously.

[0017] In the external parameter calibration between IMU and camera, the rotation matrix of the spatial external parameter between IMU and camera is jointly optimized by maximum likelihood estimation and translation vectors Based on the rotation matrix and translation vectors Further construct the coordinate system transformation matrix T between IMU and camera imu ;

[0018] The IMU branch expansion module includes:

[0019] IMU data pre-processing unit, used for denoising and calibrating multiple different IMU data through Kalman filtering;

[0020] The IMU state vector construction unit is used to construct the IMU total state vector. The IMU total state vector is:

[0021]

[0022] where X I is the state of the IMU at time k, X calib_C X is the conversion external parameter between IMU and camera, C The local IMU clone state in the cloned sliding window includes the past m image times;

[0023] IMU time synchronization unit is used to synchronize the camera time offset t dC ,pass Align camera time with IMU time;

[0024] The multimodal data fusion unit is respectively transformed by the coordinate system matrix T imu The angular velocity and acceleration data of the unmanned boat collected by the IMU are mapped to the multi-view camera image feature space. The fully connected layer is used to encode the IMU total state vector into a feature vector matching the number of image feature channels, and then spliced ​​and fused with the image features of each view along the channel dimension.

[0025] Preferably, the multimodal data fusion unit is configured to perform the following process:

[0026] Generate the IMU feature vector using the fully connected layer weight matrix W and the bias vector b:

[0027]

[0028] Next, we transform the dimension C img *1*1 F imu With dimension C img *H img *W img The image feature F img Splicing, get the dimension (C img +C imu )*H img *W img The fusion feature F fused .

[0029] Preferably, the BEV environment modeling module includes:

[0030] Temporal self-attention unit, used to extract temporal information from historical BEV features through BEV Queries;

[0031] Spatial attention unit, used to extract spatial information from multi-view image feature maps;

[0032] 6-layer stacked encoder, used to fuse spatiotemporal features layer by layer to generate the BEV feature map at the current moment;

[0033] A loss function design module is used to add IMU-related loss function terms to the loss function used to evaluate model predictions.

[0034] Optimized 3D inspection heads for:

[0035] By introducing the perspective supervision loss L pers ,enhancing the multi-view geometric consistency learning capability;

[0036] By improving the time encoder, it is used to eliminate the feature offset caused by the movement of the unmanned boat.

[0037] Preferably, the loss function design module is used to process the following steps:

[0038] IMU feature fusion loss Among them, BEV pred It is the predicted BEV feature that integrates IMU information. true is the real BEV feature. This loss function measures the effect of the fusion of IMU encoding features and image features, ensuring that IMU information is correctly integrated into the BEV feature.

[0039] Total loss function L total :L total =L det +L seg +L imu-fuse , where L det 、L seg They are the 3D detection loss function and map segmentation loss function of the model respectively.

[0040] Preferably, the optimized 3D detection head includes:

[0041] The perspective supervision loss unit is used to set the loss weights of perspective loss and BEV loss to be equal, i.e. μ bev =μ pers =1. Then L det =μ bev L bev +μ pers L pers , where L bev is the loss function of the original 3D detection head, L pers It is perspective loss;

[0042] The time encoder improvement unit performs bilinear distortion on the historical BEV features through the transformation matrix T_k^t∈SE(3) and then concatenates them with the current features;

[0043] The residual dimensionality reduction unit performs channel dimensionality reduction on the spliced ​​multi-frame features.

[0044] Preferably, the grid map generation module is used to process the following steps:

[0045] The BEV feature map generated in the BEV environment modeling module is discretized into a matrix: G = {g ij};

[0046] Set the grid value based on the environmental perception result: g ij =1 is the passable area, g ij =0 is the obstacle area;

[0047] The grid status is updated in real time to reflect the dynamic obstacle distribution.

[0048] Preferably, the path planning module adopts an improved evaluation function

[0049] f(n)=g(n)+h(n)+D(n) is used to evaluate the optimal path node of the unmanned boat in a dynamic environment, where g(n) is the actual distance cost from the starting point to the current node, h(n) is the heuristic estimated cost from the current node to the target node, and D(n) is the hazard function introduced into the Voronoi field, which is constructed based on the dynamic Voronoi field:

[0050]

[0051] Among them, d o (i, j, t) is the distance from the current node to the obstacle, reflecting the spatial relationship between the node and the obstacle; d v (i, j, t) is the distance from the current node to the Voronoi boundary. This distance is used to measure the position relationship of the node in the Voronoi diagram and affects the distribution of the potential field. a(t)>0 is a time-dependent function used to control the potential field attenuation rate. It is the function that controls the maximum range of the potential field, the distance d from the current node to the obstacle o (i,j,t) exceeds When the potential field is significantly reduced or disappears, τ is the penalty factor, v obs (t) is the velocity function of the moving obstacles around the UAV.

[0052] Preferably, the path planning module is used to execute the path planning steps including the following:

[0053] Initialize open queue and close queue;

[0054] When iteratively processing nodes, the evaluation value of the hazard function D(n) is calculated for each adjacent node;

[0055] When r V (i,j,t)>0 or d v When (i, j, t) ≥ D, the node evaluation value is set to infinity to exclude the dangerous area where the unmanned boat is traveling;

[0056] Among them, D is the minimum distance to obstacles set according to the actual sea surface operation scenario of the unmanned boat;

[0057] Generate the final safe path by backtracking the parent node.

[0058] In a second aspect, a method for autonomously returning an unmanned boat to a port by integrating BEV technology and an optimized A* algorithm is implemented according to the system for autonomously returning an unmanned boat to a port by integrating BEV technology and an optimized A* algorithm described in the first aspect, and the processing steps include:

[0059] Step S1: Multi-source data collection and feature extraction

[0060] Capture images from multiple perspectives through multiple cameras, extract features through the MobileNetV3 neural network model, and simultaneously obtain angular velocity and acceleration data from the IMU;

[0061] Step S2: Camera and IMU calibration

[0062] By calibrating the internal and external parameters of each camera, calculating the external parameter relationship between multiple cameras through the perspective n-point algorithm, and jointly optimizing the external parameters between the IMU and each camera through maximum likelihood estimation, we can further obtain the transformation matrix T between the camera coordinate system and the IMU coordinate system. imu ;

[0063] Step S3: The IMU data is fused with the feature maps of each view to denoise the IMU data and construct the state vector. After time synchronization, the transformation matrix T is used to imu Mapping to the feature space of each view image, and then splicing according to the channel dimension to obtain a multi-view feature map that integrates IMU data;

[0064] Step S4: BEV environment modeling

[0065] The fused multi-view image feature map data is input into the BEV model, which is sequentially passed through the temporal self-attention unit and the spatial attention unit, and then stacked in six layers to generate the BEV feature map. imu-fuse Designing the total loss function of the BEV model is conducive to iterative optimization of the BEV model during training; the 3D detection head in the BEV model introduces the perspective supervision loss L pers , enhance the multi-view geometric consistency learning ability and improve the time encoder to eliminate the feature offset caused by the movement of the unmanned boat;

[0066] Step S5: Dynamic grid map generation discretizes BEV features into a grid map and marks the traversable areas and obstacles in real time;

[0067] Step S6: Safe path planning

[0068] Based on the grid graph, the optimized A* algorithm is adopted to dynamically eliminate dangerous nodes and backtrack to generate a safe return path by introducing the hazard function D(n) containing a dynamic Voronoi field.

[0069] Compared with the prior art, the present invention has the following beneficial effects:

[0070] 1. This invention provides an autonomous return-to-port system and method for unmanned watercraft (UUV) that integrates BEV technology and an optimized A* algorithm. Its design aims to innovatively apply BEV technology, widely used in the field of autonomous driving, to the field of UUVs. To address the issue of wave-induced pitch motion during UUV operations, which causes changes in camera perspective and makes it difficult to stably extract multi-view image features, an IMU branch is expanded within the BEV model (i.e., an IMU branch expansion module is added) to integrate inertial IMU data, such as the UUV's angular velocity and acceleration.

[0071] At the same time, the perspective supervision loss L is introduced in the 3D detection head of the BEV model pers , to enhance the multi-view geometric consistency learning ability, and improve the temporal encoder to eliminate feature offset caused by the motion of the unmanned vehicle.

[0072] In addition, the A* algorithm is optimized by embedding a dynamic Voronoi field and combining a speed penalty factor This solves the problem that the traditional A* path planning algorithm only uses path length as a heuristic function, which causes the planned path to be too close to obstacles.

[0073] 2. In summary, the present invention integrates BEV technology with an optimized A* algorithm to achieve efficient fusion of multimodal data. It improves the 3D detection head to enhance multi-view geometric consistency, and designs a dynamic Voronoi field risk function to optimize path planning, thereby significantly improving perception accuracy and navigation safety in complex water surface environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0075] Figure 1 This is a schematic diagram of the system structure of an unmanned boat autonomous return-to-port system that integrates BEV technology and optimized A* algorithm provided by the present invention;

[0076] Figure 2 This is a flow chart of a method for an autonomous return-to-port system for an unmanned boat that integrates BEV technology and an optimized A* algorithm, provided by the present invention. DETAILED DESCRIPTION

[0077] The following will clearly and completely describe the technical solution in this embodiment of the present invention in conjunction with the drawings in this embodiment of the present invention. Obviously, the embodiment described is only one embodiment of the present invention, not all embodiments of the present invention. Based on this embodiment of the present invention, all other embodiments of the present invention obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0078] Example 1

[0079] See also Figures 1 to 2 Embodiment 1 of the present invention provides an autonomous return-to-port system for unmanned boats that integrates BEV technology and an optimized A* algorithm, comprising:

[0080] The feature extraction module is used to capture multi-view images from multiple cameras on the unmanned boat, extract multi-view image features through a neural network model, and collect multiple different IMU data from the unmanned boat's IMU;

[0081] The specific use functions of the feature extraction module include the following steps:

[0082] First, six high-definition cameras were deployed on the bow, port front, starboard front, port rear, starboard rear, and stern of the unmanned boat.

[0083] Then, the multi-view images captured by the multiple cameras of the unmanned boat are used to extract multi-view image features through the neural network MobileNetV3 model;

[0084] Next, the UAV IMU is fixed under the UAV's deck, close to the hull's centerline and center of gravity to reduce vibration and external interference;

[0085] Finally, multiple different IMU data are collected from the unmanned boat IMU, such as the angular velocity and acceleration of the boat.

[0086] The camera and IMU calibration module is used to complete the intrinsic parameter calibration of a single camera, the extrinsic parameter calibration between multiple cameras, and the extrinsic parameter calibration between the IMU and the camera through the Zhang Zhengyou calibration method.

[0087] The specific usage functions of the camera and IMU module include the following steps:

[0088] Through the aforementioned calibration of the intrinsic parameters of a single camera of the unmanned boat, Zhang Zhengyou's calibration method is used to obtain a set of known three-dimensional spatial points and corresponding two-dimensional pixel coordinates in each camera using a checkerboard calibration plate. The image detection algorithm is used to identify corner points, and homography constraints are established to describe the mapping relationship between plane points and image points. The internal and external parameter matrices relative to the boat body are solved to achieve the conversion from the image coordinate system to the world coordinate system.

[0089] Multi-camera extrinsic calibration: A checkerboard calibration plate is fixed within the common viewing area of ​​the UAV, triggering the acquisition of calibration images from multiple cameras simultaneously. The initial transformation matrix of each camera's coordinate system is solved using the perspective n-point algorithm, and the angular velocity and acceleration data of the IMU and the image streams of multiple cameras are synchronously collected.

[0090] IMU and camera extrinsic calibration, joint optimization of the rotation matrix in the spatial extrinsic parameters between IMU and camera through maximum likelihood estimation and translation vectors Based on the rotation matrix and translation vectors Further construct the coordinate system transformation matrix T between IMU and camera imu ;

[0091] Therefore, the camera and IMU calibration module is used to calibrate the internal parameters between single cameras, the external parameters between multiple cameras, and the conversion matrix T between the camera coordinate system and the IMU coordinate system. imu , ensuring the spatiotemporal consistency of data fusion between cameras with different perspectives and between cameras and IMU.

[0092] IMU branch expansion module, the IMU branch expansion module is a new module in the BEV environment modeling module. By adding the IMU branch to process angular velocity, acceleration and other data, the angular velocity, acceleration and other information of the ship in the IMU data are used to assist in the motion estimation of the ship.

[0093] It includes an IMU data preprocessing unit, an IMU state vector construction unit, an IMU time synchronization unit, and a multimodal data fusion unit. The multimodal data fusion unit is used to map the fused IMU data to the multi-view image feature space through the coordinate system transformation relationship, thereby realizing the fusion of multiple different IMU data;

[0094] Wherein, the IMU branch expansion module includes:

[0095] The IMU data preprocessing unit is used to denoise and calibrate multiple different IMU data through Kalman filtering. The specific processing process is: collecting the UAV IMU data (UAV angular velocity, acceleration, etc.) at a specific Hz, denoising it through Kalman filtering, and calibrating it according to the IMU characteristics and installation conditions to ensure data accuracy;

[0096] IMU state vector construction unit, after the IMU data is filtered, denoised and calibrated, the IMU state vector construction unit is used to construct the IMU total state vector. The IMU total state vector is:

[0097] where X I is the state of the IMU at time k, X calib_C X is the conversion external parameter between IMU and camera, C The local IMU clone state in the cloned sliding window includes the past m image times;

[0098]

[0099] (in Indicates that at t k At this moment, the IMU moves from the world coordinate system {G} to the IMU local coordinate system I k rotation. and Represents the velocity and t of the IMU in the world coordinate system k The position at the moment. b g and b a is the bias of the gyroscope and accelerometer. and Represents the rigid body transformation between the camera coordinate system {C} and the IMU coordinate system {I}. )

[0100] IMU time synchronization unit is used to synchronize the camera time offset t dC ,pass The camera time is aligned with the IMU time. The specific processing process is as follows: IMU data and view image features are synchronized in time: based on the IMU time, the camera time offset t dC ,pass (t C Time synchronization for cameras (reporting time);

[0101] Multimodal data fusion unit, used to transform the coordinate system matrix T imu The angular velocity, acceleration and other data of the unmanned boat measured by the IMU are mapped to the feature space of the current frame multi-view image. The fully connected layer is used to encode the IMU total state vector into a feature vector that matches the number of feature channels of the current frame multi-view image, and then spliced ​​and fused along the channel dimension.

[0102] The multimodal data fusion unit is used to perform the following processes:

[0103] The specific process of fusing the angular velocity, acceleration and other data of the current unmanned boat measured by the IMU with the image features of one of the viewpoints is as follows:

[0104] Generate the IMU feature vector using the fully connected layer weight matrix W and the bias vector b:

[0105]

[0106] Next, we transform the dimension C img *1*1 F imu With dimension C img *H img *W img The current frame and current view image feature F img Splicing, get the dimension (C img +C imu )*H img *W img The fusion feature F of the current frame and current perspective fused .

[0107] The IMU data performs the above fusion operations with the feature maps of the remaining viewpoint images of the current frame respectively.

[0108] More specifically, the multimodal data fusion unit realizes the fusion of IMU data and the multi-view image features of the current frame through the following processing:

[0109] First, align the data coordinate systems of different modes: according to the relationship between IMU and camera external parameters The IMU attitude information is converted into the spatial transformation of the image feature map of the current frame and current perspective. The IMU attitude change matrix is ​​T imu (Constructed by rotation matrix and translation vector), then the feature map F of the current frame image img The spatial transformation of a point (x, y) is (x', y') = T img .(x,y), so that the IMU information and the view image features are expressed in the same coordinate system;

[0110] Match the IMU data to the dimension of the image feature: Use a fully connected layer to transform the IMU state vector (speed acceleration etc.) is encoded into a feature vector F with the same number of channels as the image feature map imu , the weight matrix of the fully connected layer is W, and the bias vector is b, then After encoding F imu Dimension C img *1*1;

[0111] Combine the IMU data vector with the image data vector of the current frame and current perspective: F fused =[F img ,F imu ], about to F imu Along the channel dimension and F img Splicing. F img Dimension C img *H img *W img ,F imu Dimension C img *1*1, after splicing F fused The dimension is (C img +C imu )

[0112] *H img *W img ;

[0113] The IMU data is fused with the image feature maps of the other viewpoints of the current frame to obtain the multi-view image features of the current frame that integrate the angular velocity, acceleration and other data of the unmanned boat.

[0114] The BEV environment modeling module is used to input the multi-view image features of the current frame and the fused IMU data into the BEV model to generate a BEV feature map representing the unmanned boat environment;

[0115] The BEV environment modeling module includes:

[0116] Temporal self-attention unit, used to extract temporal information from historical BEV features through BEV Queries;

[0117] Spatial attention unit, used to extract spatial information from multi-view image feature maps;

[0118] 6-layer stacked encoder, used to fuse spatiotemporal features layer by layer to generate the BEV feature map at the current moment;

[0119] Loss function design module. After the IMU branch extension module is expanded in the BEV model, a novel loss function design module is needed to evaluate the accuracy of the BEV model in generating the BEV feature map that represents the unmanned boat environment. Therefore, the IMU-related loss function term is added to the total loss function of the BEV model.

[0120] The loss function design module is used to process the following steps:

[0121] IMU feature fusion loss Among them, BEV pred It is the predicted BEV feature that integrates IMU information. true It is the real BEV feature. This loss function measures the effect of the fusion of IMU encoding features and image features, ensuring that the IMU information is correctly integrated into the BEV feature.

[0122] Total loss function L total :L total =L det +L seg +L imu-fuse , where L det 、L seg They are the 3D detection loss function and map segmentation loss function of the model, where

[0123]

[0124] (where C is the number of categories, P is the set of pixels, y c (p) is the true label, is the predicted label)

[0125] L det =L cls +L loc +L ori +L size +L vel +L pers

[0126] Classification loss Where N is the number of bounding boxes, C is the number of categories, and y ic is the true value of the cth class of the i-th bounding box, is the predicted probability.

[0127]

[0128] Where (x, y, z) are the coordinates of the center of the 3D bounding box.

[0129] Direction loss

[0130] Where θ is the heading angle of the USV.

[0131] Size loss

[0132] where (τ, w, h) are the length, width, and height of the bounding box.

[0133] Speed ​​loss

[0134] in is the predicted speed, It's the real speed.

[0135] The total loss function is designed based on a combination of detection loss, segmentation loss, and IMU feature fusion loss. By calculating the error between the model's predictions and the true values, it guides the BEV model's optimization during training, thereby improving the UAV's perception accuracy and navigation safety in complex marine environments. In existing models, the 3D detection head lacks geometric constraints during multi-view projection and lacks the ability to learn geometric consistency across multiple perspectives, reducing the accuracy of demarcation between traversable and obstacle areas. Therefore, existing technologies suffer from a core flaw: weak geometric constraints on the 3D detection head.

[0136] This embodiment uses an optimized 3D detection head to introduce perspective supervision loss L pers , enhance the multi-view geometric consistency learning ability, and further eliminate the idiosyncratic offset caused by the movement of the unmanned boat;

[0137] Among them, the perspective supervision loss unit is used to set the loss weights of perspective loss and BEV loss to be equal, that is, μ bev =μ pers =1. Then L det =μ bev L bev +μ pers L pers , where L bev is the loss function of the original 3D detection head, L pers It is perspective loss;

[0138] The time encoder improvement unit performs bilinear distortion on the historical BEV features through the transformation matrix T_k^t∈SE(3) and then concatenates them with the current features;

[0139] The residual dimension reduction unit performs channel dimension reduction on the spliced ​​multi-frame features;

[0140] Improved time encoder to eliminate feature offset caused by UAV motion.

[0141] More specifically, the BEV environment modeling module is used to perform the following process: after fusing the IMU data with the image data, a multi-scale feature map of multiple viewpoint images of the unmanned vehicle that incorporates the IMU information is obtained at time t: in is the Feature of the i-th view camera, N view is the total number of camera views, and at the same time, the BEV Features B at time t-1 must be retained t-1 In each Encoder layer, first use BEV Queries Q from B through TSA (temporal self-attention) module t-1 The temporal information is queried and integrated to obtain the modified BEV Queries Q′. Then, in the same Encoder layer, the BEV Queries Q′ “corrected” by TSA is obtained from the multi-camera FeaturesF through the SCA (spatial attention) module. t The spatial information is queried and integrated to obtain the further modified BEV Queries Q". This encoder layer performs FF calculation on the twice-modified BEV features Q" (BEV Queries) and then outputs it as the input of the next encoder layer. Six layers are stacked in this way, that is, after six rounds of fine-tuning, the unified BEV feature map B at time t is generated. t , and then input the optimized 3D detection head.

[0142] Improved 3D inspection head performance helps generate more accurate BEV representations, addressing the low perception accuracy of the original 3D inspection head in complex dock environments:

[0143] To enhance the UAV's ability to learn multi-view geometric consistency and thus improve the perception accuracy of the 3D detection head in complex scenes, perspective supervision is introduced through auxiliary perspective loss:

[0144]

[0145] The 3D bounding box predicted by the model is projected onto the 2D box of the k-th camera image plane as Proj k (3D Box), the real 2D box is 2D N views is the number of cameras, IOULoss measures the position deviation of the 2D box, SegLoss is the semantic segmentation loss of the projection area, γ Iou and γ seg is the weight coefficient.

[0146] The loss weights of perspective loss and BEV loss are set equal, i.e. μ bev =μ pers =1. Then L det =μ bev L bev +μ pers L pers , where L bev is the loss function of the original 3D detection head, L pers It is perspective loss.

[0147] In order to eliminate the feature offset caused by the motion of the unmanned boat and ensure the spatial alignment of the temporal features, a simple warping and connection strategy is adopted by improving the temporal encoder to give the BEV features B at different frames k. k , first according to the reference frame transformation matrix between frame t and frame k Among them, SE(3) is a special Euclidean group, and B k Bilinear warping to the current frame as Then, the previous BEV features are concatenated with the current BEV features along the channel dimension and a residual block is used for dimensionality reduction.

[0148] A grid map generation module is used to divide the BEV feature map generated by the BEV environment modeling module into grid maps of passable areas and obstacle areas;

[0149] The grid image generation module is used to process the following steps:

[0150] The BEV feature map generated by the BEV environment modeling module is discretized into a matrix: G = {g ij};

[0151] Set the grid value based on the environmental perception result: g ij =1 is the passable area, g ij =0 is the obstacle area;

[0152] Update grid status in real time to reflect dynamic obstacle distribution;

[0153] The more specific processing is as follows: the bird's-eye view of the unmanned boat generated by the BEV environment modeling module is divided into several grids, each grid represents a specific area, and each grid is marked as a passable area or an obstacle area according to the environmental information of the unmanned boat. The grid map can be expressed as:

[0154] G={g ij}

[0155] Among them, g ij represents the grid at row i and column j, g ij =1 indicates the area where the unmanned boat can pass, g ij =0 indicates an obstacle area where the unmanned boat cannot pass.

[0156] The path planning module uses the improved A* algorithm based on the Voronoi field to plan the path for the unmanned boat to return to the port based on the grid map;

[0157] The path planning module uses an improved evaluation function f(n)=g(n)+h(n)+D(n) to evaluate the cost of each node and guide the search for the optimal target node on the grid map where the unmanned boat is located. The hazard function D(n) takes into account the dynamic obstacles on the water surface and is constructed based on the dynamic Voronoi field:

[0158]

[0159] Among them, d o (i, j, t) is the distance from the current node to the obstacle, reflecting the spatial relationship between the node and the obstacle; d v (i, j, t) is the distance from the current node to the Voronoi boundary. This distance is used to measure the position relationship of the nodes in the Voronoi diagram and affects the distribution of the potential field. a(t)>0 is a time-dependent function used to control the potential field decay rate. It is the function that controls the maximum range of the potential field, the distance d from the current node to the obstacle o (i,j,t) exceeds When , the influence of the potential field will be significantly reduced or disappear; τ is the penalty factor, v obs (t) is the velocity function of the moving obstacles around the UAV.

[0160] The path planning module is used to perform the following steps for path planning:

[0161] Initialize open queue and close queue;

[0162] When iteratively processing nodes, the evaluation value of the hazard function D(n) is calculated for each adjacent node;

[0163] When rV (i,j,t)>0 or d v When (i, j, t) ≥ D, the node evaluation value is set to infinity to exclude the dangerous area where the unmanned boat is traveling;

[0164] Among them, D is the minimum distance to obstacles set according to the actual sea surface operation scenario of the unmanned boat;

[0165] Generate the final safe path by backtracking the parent node;

[0166] More specifically, the path planning module is used to perform the following processes:

[0167] Write a basic path planning program based on the traditional A* algorithm to achieve basic obstacle avoidance and path planning capabilities;

[0168] By introducing the hazard function D(n) of the Voronoi field to optimize the heuristic function, the path planning process can effectively avoid obstacles in a dock environment with multiple obstacles, reduce the risk of collision, and ensure the navigation safety of the unmanned boat.

[0169] The traditional A* algorithm guides the search by evaluating the function f(n) = g(n) + h(n), where g(n) represents the actual distance cost from the starting point to the current node, and h(n) is the estimated distance cost from the current node to the target node. During the search, the algorithm compares the f(n) values ​​of each node and selects the optimal path node for expansion. The specific workflow is as follows: Initialize two queues: an open queue (open list) and a closed queue (closed list). The open queue is used to store nodes to be processed, and the closed queue is used to store processed nodes. The starting node is added to the open queue, and the node n with the smallest f(n) value is taken from the open queue as the current node for processing. If the current node n is the target node, the path search is successful, and the path is backtracked according to the pointer. Otherwise, the current node n is moved from the open queue to the closed queue, and the generation of adjacent nodes of the current node n begins. For each adjacent node, the following operations are performed:

[0170] If the adjacent node is not passable (such as an obstacle) or is already in the closed queue, the node is skipped.

[0171] If the adjacent node is not in the open queue, add it to the open queue and calculate its g(n), h(n) and f(n) values. At the same time, set the current node n as the parent node of the adjacent node.

[0172] If the adjacent node is already in the open queue, check whether the path through the current node n to reach the adjacent node is better (that is, whether the g(n) value is smaller). If so, update the g(n), h(n) and f(n) values ​​of the adjacent node and set the current node n as the parent node of the adjacent node.

[0173] Repeat the above steps until the open queue is empty (indicating that no path can be found) or the target node is found.

[0174] The A-star algorithm efficiently finds the optimal path by continuously updating the node heuristic function f(n), prioritizing the nodes most likely to be close to the target for expansion. However, this traditional algorithm only uses path length as a heuristic function, which may cause the planned path to be too close to obstacles, posing a safety hazard to the unmanned ship's navigation.

[0175] In order to solve the problem of the traditional A* algorithm being too close to obstacles in the navigation path planning of unmanned ships, the evaluation function of the A* algorithm is improved by using the concept of establishing a navigation boundary based on the Voronoi field algorithm. Its definition is as follows:

[0176] f(n)=g(n)+h(n)+D(n)

[0177] Among them, g(n) is the actual distance cost from the starting point to the current node, h(n) is the heuristic estimated cost from the current node to the target node, and D(n) is the hazard function introduced into the Voronoi field.

[0178] By embedding dynamic factors such as the speed of moving obstacles on the sea surface into the field formula, the dynamic Voronoi field is defined as follows:

[0179]

[0180] Among them, d o (i, j, t) is the distance from the current node to the obstacle, reflecting the spatial relationship between the node and the obstacle; d v (i, j, t) is the distance from the current node to the Voronoi boundary. This distance is used to measure the position relationship of the nodes in the Voronoi diagram and affects the distribution of the potential field. a(t)>0 is a time-dependent function used to control the potential field decay rate. It is the function that controls the maximum range of the potential field, the distance d from the current node to the obstacle o (i,j,t) exceeds When , the influence of the potential field will be significantly reduced or disappear; τ is the penalty factor, v obs (t) is the speed of the moving obstacles around the USV.

[0181] During the navigation process of the unmanned boat, it is necessary not only to consider the water depth constraint, but also to maintain a certain safety distance between the generated path and obstacles. After adding the hazard function, the evaluation function of each grid is defined as follows:

[0182]

[0183] Where D is a fixed safety constraint distance, which is set according to the actual application scenario and is used to determine whether the node is within the safety range.

[0184] The advantages of the unmanned boat autonomous return-to-port system of the present invention, which integrates BEV technology and optimized A* algorithm, are as follows:

[0185] 1. Using advanced BEV technology in unmanned boats, a multi-view camera array is used to build three-dimensional perception of the water surface environment, improving the real-time perception capability of complex scenes.

[0186] 2. In view of the water surface environment in which unmanned boats operate, an IMU branch is added to the BEV model, and a state vector is designed to synchronize spatiotemporal data. The motion parameters are encoded through a fully connected layer and spliced ​​with visual features. An IMU feature fusion loss term is added to achieve deep fusion optimization of IMU and vision to solve the interference of water surface fluctuations.

[0187] 3. Improve the 3D detection head in the BEV model and introduce perspective loss supervision L pers , balance BEV and perspective space feature learning; improve the temporal encoder, align features through motion compensation, and improve the accuracy of dynamic detection.

[0188] 4. Improve the A* algorithm, integrate the dynamic Voronoi field to quantify obstacle threats, and introduce a speed penalty factor and distance attenuation coefficient, actively avoiding high-speed obstacles and ensuring navigation safety.

[0189] 5. The BEV model is combined with the unmanned boat IMU and the 3D detection head is improved to generate a BEV feature map representing the unmanned boat environment. Based on the BEV feature map as the raster map, an optimization planning algorithm based on the A* algorithm is proposed to perform the path planning task of the unmanned boat autonomous docking. The unmanned boat autonomous docking system can perceive the marine environment in real time, plan the return path, and improve the accuracy, autonomy and safety of the unmanned boat's environmental perception.

[0190] Example 2

[0191] See also Figures 1 to 2 Embodiment 2 of the present invention provides a method for autonomously returning an unmanned boat to a port by integrating BEV technology and optimizing the A* algorithm. The method is implemented according to the autonomously returning an unmanned boat to a port system integrating BEV technology and optimizing the A* algorithm described in embodiment 1. The processing steps include:

[0192] S1: Multi-source data collection and feature extraction

[0193] Multiple cameras are used to capture images from multiple viewpoints and extract features using the MobileNetV3 neural network model. The angular velocity, acceleration, and other data from the IMU are also acquired simultaneously. The details are as follows:

[0194] S11: Deploy six high-definition cameras on the bow, port front, port front, port rear, port rear and stern of the unmanned boat to simultaneously capture multi-view images.

[0195] S12: Use the MobileNetV3 neural network model to extract multi-scale feature maps of images from each perspective and collect angular velocity and acceleration data from the IMU.

[0196] S13: Fix the IMU to the centerline below the deck, perform denoising and calibration through Kalman filtering, and output the filtered data.

[0197] S2: Camera and IMU calibration

[0198] S21: Single-camera intrinsic calibration: Using a checkerboard calibration plate, the Zhang Zhengyou calibration method is used to solve the camera's internal and external parameter matrices relative to the hull, achieving the transformation from the world coordinate system to the image coordinate system. S22: Multi-camera extrinsic calibration: A checkerboard is fixed in the common view area, and the perspective n-point algorithm is used to solve the initial transformation matrix between each camera.

[0199] S23: IMU and camera extrinsic calibration: synchronously collect IMU data and image streams, and jointly optimize the rotation matrix and translation vectors Get the transformation matrix T imu .

[0200] S3: IMU data is fused with the feature maps of each view image

[0201] S31: Construct the IMU total state vector X, which includes the IMU pose, external parameters and clone state. The formula is:

[0202] S32: Time synchronization: based on time offset t dC Align IMU and camera data,

[0203] S33: Multimodal Fusion: Through T imu Map the IMU data to the image feature space, encode it in the fully connected layer, and splice it along the channel dimension to generate a fusion feature map F fused .

[0204] S34: The image feature maps of different perspectives at the current moment are respectively fused with the IMU data at the current moment to obtain a multi-perspective fusion feature map fused with the IMU data at the current moment.

[0205] S4: BEV Environmental Modeling

[0206] S41: Input the multi-view fusion feature map fused with IMU data at the current moment into the BEV model.

[0207] S42: The BEV model uses a 6-layer stacked encoder, a temporal self-attention unit to extract historical BEV features, and a spatial attention unit to aggregate multi-view features to generate the BEV feature map at the current moment, which is finally input into the optimized 3D detection head.

[0208] S43: The specific optimization of the 3D detection head is as follows:

[0209] S44: Introduce perspective supervision loss: Project the 3D detection frame to the image plane of each view and calculate the perspective loss L pers With BEV loss L bev , the weight is set to 1:1.

[0210] S45: Improved temporal encoder: Use the transformation matrix T_k^t∈SE(3) to perform bilinear warping on the historical BEV features, and then reduce the dimension after splicing to eliminate motion offset.

[0211] S5: Dynamic grid map construction

[0212] S51: Discretize BEV features into a grid matrix G = {g ij}, the grid size is set according to the environment resolution. S52: Mark grid status: g ij =1 is the passable area, g ij =0 is the obstacle area, and the dynamic obstacle distribution is updated in real time.

[0213] S6: Safe Path Planning

[0214] S61: Initialize the open queue and the closed queue, and set the starting point and the target point.

[0215] S62: Calculate the node evaluation function f(n)=g(n)+h(n)+D(n), where D(n) is based on the dynamic Voronoi field formula:

[0216]

[0217] S63: Eliminate dangerous nodes: If r V (i,j,t)>0&or d v(i, j, t) ≥ D, let D(n) = ∞, then f(n) = ∞, avoiding the path close to the obstacle. Where D is the minimum safe distance between the unmanned boat and the dynamic obstacle, which is set according to the actual application.

[0218] S64: Backtrack the parent node to generate a safe path and output the optimal trajectory for the unmanned boat to return to the port.

[0219] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An autonomous return-to-port system for unmanned boats that integrates BEV technology and optimized A* algorithm, characterized by: It includes: The feature extraction module is used to capture multi-view images from multiple cameras of the unmanned boat, extract multi-view image features through a neural network model, and collect multiple different IMU inertial data from the unmanned boat's inertial measurement unit (IMU); The camera and IMU calibration module is used to complete the internal and external parameter calibration of each camera, the external parameter calibration between multiple cameras, and the external parameter calibration between the IMU and the camera; The IMU branch expansion module includes an IMU data preprocessing unit, an IMU state vector construction unit, an IMU time synchronization unit, and a multimodal data fusion unit. The multimodal data fusion unit is used to map the collected IMU data to the multi-view image feature space through the coordinate system transformation relationship, and then fuse the IMU data with the feature maps of each view image; The BEV environment modeling module is used to input the multi-view image features and the fused IMU data into the BEV environment modeling module to generate a BEV feature map representing the unmanned boat environment; A grid map generation module is used to divide the BEV feature map generated by the BEV environment modeling module into grid maps of passable areas and obstacle areas; The path planning module uses the improved A* algorithm based on the Voronoi field to plan the path for the unmanned boat to return to the port based on the grid map.

2. The autonomous return-to-port system for unmanned boats integrating BEV technology and optimized A* algorithm according to claim 1 is characterized in that: The camera and IMU calibration module is used to perform the following process: In the calibration of a single camera, Zhang Zhengyou calibration method is used to obtain 3D spatial points and 2D pixel coordinates through a checkerboard calibration plate, identify corner points, establish homography constraints, solve internal and external parameter matrices, and realize the conversion from image coordinate system to world coordinate system. In the external parameter calibration between multiple cameras, a checkerboard calibration plate is fixed in the common viewing area of ​​the unmanned boat, and multiple cameras are triggered to collect calibration images at the same time. The initial transformation matrix of each camera coordinate system is solved by the perspective n-point algorithm; the angular velocity and acceleration data of the IMU and the image stream of multiple cameras are collected synchronously. In the external parameter calibration between IMU and camera, the rotation matrix of the spatial external parameter between IMU and camera is jointly optimized by maximum likelihood estimation and translation vectors Based on the rotation matrix and translation vectors Further construct the coordinate system transformation matrix T between IMU and camera imu ; The IMU branch expansion module includes: IMU data pre-processing unit, used for denoising and calibrating multiple different IMU data through Kalman filtering; The IMU state vector construction unit is used to construct the IMU total state vector. The IMU total state vector is: where X I is the state of the IMU at time k, X calib_C X is the conversion external parameter between IMU and camera, C The local IMU clone state in the cloned sliding window includes the past m image times; IMU time synchronization unit is used to synchronize the camera time offset t dC ,pass Align camera time with IMU time; The multimodal data fusion unit is respectively transformed by the coordinate system matrix T imu The angular velocity and acceleration data of the unmanned boat collected by the IMU are mapped to the multi-view camera image feature space. The fully connected layer is used to encode the IMU total state vector into a feature vector matching the number of image feature channels, and then spliced ​​and fused with the image features of each view along the channel dimension.

3. The autonomous return-to-port system for unmanned boats integrating BEV technology and optimized A* algorithm according to claim 1 or 2, characterized in that: The multimodal data fusion unit is used to perform the following process: Generate the IMU feature vector using the fully connected layer weight matrix W and the bias vector b: Next, we transform the dimension C img *1*1 F imu With dimension C img *H img *W img The image feature F img Splicing, get the dimension (C img +C imu )*H img *W img The fusion feature F fused .

4. The autonomous return-to-port system for unmanned boats integrating BEV technology and optimized A* algorithm according to claim 1 is characterized in that: The BEV environment modeling module includes: Temporal self-attention unit, used to extract temporal information from historical BEV features through BEV Queries; Spatial attention unit, used to extract spatial information from multi-view image feature maps; 6-layer stacked encoder, used to fuse spatiotemporal features layer by layer to generate the BEV feature map at the current moment; A loss function design module is used to add IMU-related loss function terms to the loss function used to evaluate model predictions. Optimized 3D inspection heads for: By introducing the perspective supervision loss L pers ,enhancing the multi-view geometric consistency learning capability; By improving the time encoder, it is used to eliminate the feature offset caused by the movement of the unmanned boat.

5. The autonomous return-to-port system for unmanned boats integrating BEV technology and optimized A* algorithm according to claim 4 is characterized in that: The loss function design module is used to process the following steps: IMU feature fusion loss Among them, BEV pred It is the predicted BEV feature that integrates IMU information. true is the real BEV feature. This loss function measures the effect of the fusion of IMU encoding features and image features, ensuring that IMU information is correctly integrated into the BEV feature. Total loss function L total :L total =L det +L seg +L imu-fuse , where L det , L seg They are the 3D detection loss function and map segmentation loss function of the model respectively.

6. The autonomous return-to-port system for unmanned boats integrating BEV technology and optimized A* algorithm according to claim 4 is characterized in that: The optimized 3D detection head includes: The perspective supervision loss unit is used to set the loss weights of perspective loss and BEV loss to be equal, i.e. μ bev =μ pers =1. Then L det =μ bev L bev +μ pers L pers , where L bev is the loss function of the original 3D detection head, L pers It is perspective loss; The time encoder improvement unit performs bilinear distortion on the historical BEV features through the transformation matrix T_k^t∈SE(3) and then concatenates them with the current features; The residual dimensionality reduction unit performs channel dimensionality reduction on the spliced ​​multi-frame features.

7. The autonomous return-to-port system for unmanned boats integrating BEV technology and optimized A* algorithm according to claim 1 is characterized in that: The grid map generation module is used to process the following steps: The BEV feature map generated in the BEV environment modeling module is discretized into a matrix: G = {g ij }; Set the grid value based on the environmental perception result: g ij =1 is the passable area, g ij =0 is the obstacle area; The grid status is updated in real time to reflect the dynamic obstacle distribution.

8. The autonomous return-to-port system for unmanned boats integrating BEV technology and optimized A* algorithm according to claim 1 is characterized in that: The path planning module uses an improved evaluation function f(n)=g(n)+h(n)+D(n) to evaluate the optimal path nodes of the unmanned boat in a dynamic environment, where g(n) is the actual distance cost from the starting point to the current node, h(n) is the heuristic estimated cost from the current node to the target node, and D(n) is a hazard function introduced into the Voronoi field, which is constructed based on the dynamic Voronoi field: Among them, d o (i, j, t) is the distance from the current node to the obstacle, reflecting the spatial relationship between the node and the obstacle; d v (i, j, t) is the distance from the current node to the Voronoi boundary. This distance is used to measure the position relationship of the node in the Voronoi diagram and affects the distribution of the potential field. a(t)>0 is a time-dependent function used to control the potential field attenuation rate. It is the function that controls the maximum range of the potential field, the distance d from the current node to the obstacle o (i,j,t) exceeds When the potential field is significantly reduced or disappears, τ is the penalty factor, v obs (t) is the velocity function of the moving obstacles around the UAV.

9. The autonomous return-to-port system for unmanned boats integrating BEV technology and optimized A* algorithm according to claim 8 is characterized in that: The path planning module is used to perform the following steps for path planning: Initialize open queue and close queue; When iteratively processing nodes, the evaluation value of the hazard function D(n) is calculated for each adjacent node; When r V (i,j,t)>0 or d v When (i, j, t) ≥ D, the node evaluation value is set to infinity to exclude the dangerous area where the unmanned boat is traveling; Among them, D is the minimum distance to obstacles set according to the actual sea surface operation scenario of the unmanned boat; Generate the final safe path by backtracking the parent node.

10. A method for autonomous return of unmanned boats to port by integrating BEV technology and optimized A* algorithm, characterized in that: It is implemented by an unmanned boat autonomous return to port system integrating BEV technology and optimized A* algorithm according to claims 1-9, and its processing steps include: Step S1: Multi-source data collection and feature extraction Capture images from multiple perspectives through multiple cameras, extract features through the MobileNetV3 neural network model, and simultaneously obtain angular velocity and acceleration data from the IMU; Step S2: Camera and IMU calibration By calibrating the internal and external parameters of each camera, calculating the external parameter relationship between multiple cameras through the perspective n-point algorithm, and jointly optimizing the external parameters between the IMU and each camera through maximum likelihood estimation, we can further obtain the transformation matrix T between the camera coordinate system and the IMU coordinate system. imu ; Step S3: IMU data is fused with the feature maps of each view image respectively Denoise the IMU data and construct the state vector. After time synchronization, pass the transformation matrix T imu Mapping to the feature space of each view image, and then splicing according to the channel dimension to obtain a multi-view feature map that integrates IMU data; Step S4: BEV environment modeling The fused multi-view image feature map data is input into the BEV model, which is sequentially passed through the temporal self-attention unit and the spatial attention unit, and then stacked in six layers to generate the BEV feature map. imu-fuse Designing the total loss function of the BEV model is conducive to iterative optimization of the BEV model during training; the 3D detection head in the BEV model introduces the perspective supervision loss L pers , enhance the multi-view geometric consistency learning ability and improve the time encoder to eliminate the feature offset caused by the movement of the unmanned boat; Step S5: Dynamic grid map generation Discretize BEV features into a grid map and mark traversable areas and obstacles in real time; Step S6: Safe path planning Based on the grid graph, the optimized A* algorithm is adopted to dynamically eliminate dangerous nodes and backtrack to generate a safe return path by introducing the hazard function D(n) containing a dynamic Voronoi field.

Images