Unmanned decision-making method and device for amphibious vehicle based on environment perception deduction

Abstract

This invention discloses an unmanned driving decision-making method and device for amphibious vessels based on environmental perception and simulation. The method includes: acquiring multimodal sensor data and processing it to obtain carrier state information, dynamic obstacle information, and environmental characteristic parameters, and encapsulating these with timestamps to form a unified environmental representation; inputting the unified environmental representation, the vessel's physical state, and mission command text into a trained amphibious vessel unmanned driving decision-making model; first, generating candidate strategies through a strategy generation module; then, dynamically simulating each candidate strategy using a world model to generate a quantitative evaluation report; inputting the quantitative evaluation report into a decision arbitration module to generate a recommended strategy and driving according to the recommended strategy; during the water-to-land transition, generating candidate landing points based on environmental characteristic parameters; evaluating and performing mechanical simulations on each candidate landing point and recommended strategy to determine the optimal landing point and optimal strategy, and generating a phased control command sequence. This invention maintains stable decision-making performance.

Description

Technical Field

[0001] This invention relates to the field of intelligent driving, specifically to a method and device for unmanned driving decision-making of amphibious vessels based on environmental perception and inference. Background Technology

[0002] With the rapid development of unmanned driving technology, its application prospects in the field of amphibious vessels are becoming increasingly broad. However, current mainstream unmanned vessel control systems still face multiple key technical bottlenecks when dealing with complex, dynamic, and extreme water-land interaction scenarios, mainly reflected in the following aspects:

[0003] (1) At the perception and decision-making level, existing systems generally rely on "black box" models such as deep learning for environmental understanding and path planning. Although these data-driven methods perform well under normal operating conditions, their internal decision-making mechanisms lack interpretability. When encountering rare extreme operating conditions (such as sudden surges caused by super typhoons, abnormal water flow vortices, etc.), it is difficult to trace the basis of the decision, making it difficult to assess and control safety risks. At the same time, although traditional classification methods based on predefined rules or shallow machine learning have interpretability, their ability to model complex environmental semantics (such as water flow structure, ship intentions, tidal dynamics) is limited, making it difficult to achieve accurate understanding of dynamic environments.

[0004] (2) The lack of physical law modeling is a fundamental defect of current pure data-driven models. The motion behavior of unmanned ships is highly dependent on natural physical laws such as hydrodynamics, wind field evolution, and sediment deposition. However, existing models mostly learn statistical correlations from historical data and lack explicit modeling of physical mechanisms such as waterway hydrodynamics, typhoon wind field dynamics, and tidal flat friction characteristics. This leads to the path planning generated by the models often violating basic hydrodynamic common sense. For example, when landing on tidal flats, they may predict the wrong hull attitude, underestimate the wave impact force, or misjudge the resistance change at the dry-wet interface, which seriously weakens the robustness of the system in complex boundary scenarios.

[0005] (3) Lack of forward-looking prediction capability in decision-making. Most mainstream systems currently adopt a "reactive control" strategy, which only makes immediate responses based on the current sensor status. They lack medium- and long-term simulation and prediction of the consequences of actions, and cannot quickly and accurately simulate the subsequent evolution of different decision paths (such as whether to get stuck in shallow water or encounter a secondary surge after avoidance). As a result, they miss the optimal decision opportunity and are prone to performance degradation or failure when facing undefined scenarios.

[0006] (4) In terms of media switching control, traditional amphibious systems rely on preset programs or manual intervention when switching media, which makes it difficult to adapt to complex tidal flats and hydrological conditions. Summary of the Invention

[0007] The purpose of this application is to propose an unmanned driving decision-making method and device for amphibious ships based on environmental perception and inference, which addresses the aforementioned technical problems.

[0008] In a first aspect, the present invention provides an unmanned driving decision-making method for amphibious vessels based on environmental perception and simulation, comprising the following steps:

[0009] An unmanned decision-making model for amphibious ships is constructed and trained to obtain a trained unmanned decision-making model for amphibious ships. The unmanned decision-making model for amphibious ships includes a world model and a policy generation module and a decision arbitration module based on a pre-trained large language model.

[0010] The system acquires and preprocesses multimodal sensor data collected at the current moment to obtain carrier state information, dynamic obstacle information, and environmental feature parameters; it then encapsulates the carrier state information, dynamic obstacle information, and environmental feature parameters with a timestamp into a unified environmental representation.

[0011] The unified environment representation, the physical state of the ship maintained in the world model, and the mission command text are input into the trained amphibious vehicle unmanned decision-making model. First, the strategy generation module generates a strategy framework, which includes several candidate strategies. Then, the world model is used to perform dynamic simulation on each candidate strategy to generate a quantitative evaluation report. The quantitative evaluation report is input into the decision arbitration module, and a preset safety rule base and mission objective weights are loaded to generate a recommended strategy and drive according to the recommended strategy.

[0012] During the water-land transition, hydrological analysis is performed based on environmental characteristic parameters to generate a list of candidate landing points. Each candidate landing point and recommended strategy in the list is evaluated and mechanically simulated to determine the optimal landing point and optimal strategy. Based on the optimal strategy and optimal landing point, a phased control command sequence is generated. The carrier is controlled based on the phased control command sequence, and it is determined whether the phase completion conditions are met or an abnormal state is triggered. If so, the decision-making process of switching to the next phase is initiated or a global replanning is triggered.

[0013] Preferably, the multimodal sensor data includes data collected by lidar, millimeter-wave radar, vision camera, IMU / GPS integrated navigation unit, sonar and environmental sensors. The IMU / GPS integrated navigation unit includes GPS, IMU and wheel speed meter. The preprocessing process includes filtering and noise reduction, distortion correction and spatiotemporal alignment to obtain the preprocessed data corresponding to each sensor.

[0014] The analysis process is as follows:

[0015] Extended Kalman filtering is used to fuse preprocessed data from GPS, IMU, and wheel speedometer to calculate the latitude, longitude, and altitude of the vessel in the world coordinate system and convert them into three-dimensional position coordinates, while outputting a three-dimensional velocity vector. Complementary filtering is used to fuse acceleration and angular velocity data from the preprocessed data corresponding to the IMU to calculate the roll angle φ and pitch angle θ, and the heading ψ is obtained by integrating the angular velocity. The vessel's state information includes three-dimensional position coordinates, three-dimensional velocity vector, roll angle φ, pitch angle θ, and heading ψ.

[0016] Point cloud data corresponding to LiDAR or millimeter-wave radar is input into a 3D target detection network to generate 3D target detection results; visual images corresponding to visual cameras are input into a 2D target detection model to obtain 2D target detection results; the 2D target detection results are projected onto 3D space through the camera model and fused with the 3D target detection results to obtain the target position, size and category;

[0017] A multi-target tracking algorithm is used to correlate each target across frames to obtain the tracking trajectory, which is then combined with the target's position, size, and category to form the target's observation state.

[0018] Particle filtering is used to smooth the target observation state over time and predict the future state to obtain dynamic obstacle information, which includes the target position, size, velocity and heading angle.

[0019] The visual image corresponding to the visual camera is input into the semantic segmentation model to obtain the semantic segmentation result. The point cloud data corresponding to the LiDAR is processed by ground segmentation and meshing to construct a digital elevation model. The semantic segmentation result is fused with the pose of the LiDAR to obtain a three-dimensional semantic point cloud. The digital elevation model is aligned with the three-dimensional semantic point cloud space to obtain a three-dimensional semantic map. The environmental feature parameters include the three-dimensional semantic map and the digital elevation model.

[0020] As a preferred method, the physical state of the ship maintained in the world model is determined using the following steps:

[0021] The current roll angle φ and bow heading ψ are combined to form the nominal physical state at the current moment. ;

[0022] The world model extrapolates the predicted physical state of the ship at the current moment based on the ship's physical state and dynamic model at the previous moment. The system calculates the difference between the nominal physical state and the predicted physical state at the current moment. If the difference exceeds a threshold, a state correction process is triggered. In the state correction process, the carrier state information is used as an observation constraint. A factor graph optimization algorithm is adopted, combined with dynamic model constraints, to smooth and backtrack the physical state within the time window corresponding to the current moment of the world model, thereby obtaining the optimized physical state at the current moment. The optimized physical state at the current moment is taken as the physical state of the ship at the current moment. If the difference does not exceed the threshold, the predicted physical state at the current moment is taken as the physical state of the ship at the current moment.

[0023] As a preferred option, the strategy framework includes action type, main objective, constraints and candidate strategies; the quantitative evaluation report includes the predicted trajectory, minimum encounter distance, maximum roll angle, estimated arrival time, energy consumption and rule risk score for each candidate strategy.

[0024] As a preferred approach, during the land-water transition process, hydrological analysis is performed based on environmental characteristic parameters to generate a candidate landing point list. Each candidate landing point and recommended strategy in the list are then evaluated and subjected to mechanical simulation to determine the optimal landing point and optimal strategy. Specifically, this includes:

[0025] Key terrain features are calculated based on the digital elevation model. These key terrain features include slope changes, protruding obstacles, and depressions. Slope changes are directly obtained by calculating the normal vector of grid vertices or the elevation difference between adjacent points. Protruding obstacles are identified by detecting positive elevation changes in a local area relative to the surrounding terrain that exceed an elevation threshold. Depressions are identified by detecting negative elevation changes in a local area that exceed an elevation threshold.

[0026] A dual-branch feature fusion-based geological hardness classification model was constructed and trained to obtain the trained geological hardness classification model. This model includes an image branch, a vibration branch, a fusion module, and a classifier. The image branch uses a pre-trained ResNet network, and the vibration branch uses a temporal Transformer encoder. Visual images and temporal data from chassis vibration sensors were input into the trained geological hardness classification model. The visual images were processed through the image branch to extract high-dimensional texture, color, and structural features. The temporal data from the chassis vibration sensors were processed through the vibration branch to extract features related to bottoming impact and resonant frequency. The high-dimensional texture, color, and structural features, as well as the features related to bottoming impact and resonant frequency, were input into the fusion module for fusion to obtain fused features. These fused features were then processed by the classifier to obtain the probability of each geological hardness category. The geological hardness was determined based on the probability of each geological hardness category.

[0027] Hydrological information is obtained by assessing nearshore water depth distribution, geological conditions, and wave impact intensity using data collected by sonar. Environmental characteristic parameters are comprehensively evaluated by integrating key topographic features, geological hardness, and hydrological information to generate a list of candidate landing points. The data for each candidate landing point includes spatial coordinates, suggested shore angle, and preliminary velocity.

[0028] In the mechanical simulation process, a multi-domain coupled dynamic model based on the fluid domain, structural domain, and ground contact domain is constructed. The fluid domain is used to describe the hydrodynamic action, the structural domain is used to describe the rigid-flexible coupling response of the carrier, and the ground contact domain is used to describe the nonlinear contact mechanical behavior between the tire or track and the terrain. The data and environmental information of each candidate landing point are input into the multi-domain coupled dynamic model to calculate the mechanical indices. Based on the mechanical indices, the optimal landing point is selected, and the recommended strategy is optimized to obtain the optimal strategy.

[0029] As a preferred option, a phased control instruction sequence is generated based on the optimal strategy and the optimal landing point, specifically including:

[0030] The optimal strategy is analyzed as a moving target, and the planned trajectory is obtained by combining the dynamic constraints of the optimal landing point;

[0031] Based on the geometric and temporal characteristics of the planned trajectory and combined with the task semantics, the planned trajectory is divided into several stages. A core control objective and a corresponding control mode are assigned to each stage, resulting in a corresponding phased control instruction sequence.

[0032] Secondly, the present invention provides an unmanned decision-making device for amphibious vessels based on environmental perception and simulation, comprising:

[0033] The model building module is configured to build and train an unmanned decision-making model for amphibious vessels, resulting in a trained unmanned decision-making model for amphibious vessels. The unmanned decision-making model for amphibious vessels includes a world model and a policy generation module and a decision arbitration module based on a pre-trained large language model.

[0034] The multimodal data processing module is configured to acquire multimodal sensor data collected at the current moment, perform preprocessing and analysis to obtain carrier state information, dynamic obstacle information and environmental feature parameters; and encapsulate the carrier state information, dynamic obstacle information and environmental feature parameters with a timestamp into a unified environmental representation.

[0035] The strategy generation module is configured to input the unified environment representation, the ship's physical state maintained in the world model, and the mission command text into the trained amphibious vehicle unmanned decision-making model. First, the strategy generation module generates a strategy framework, which includes several candidate strategies. Then, the world model is used to perform dynamic simulation on each candidate strategy to generate a quantitative evaluation report. The quantitative evaluation report is then input into the decision arbitration module, which loads the preset safety rule base and mission objective weights to generate a recommended strategy and drive according to the recommended strategy.

[0036] The planning module is configured to perform hydrological analysis based on environmental characteristic parameters during the water-land transition process, generate a list of candidate landing points, evaluate and perform mechanical simulations on each candidate landing point and recommended strategy in the list, and determine the optimal landing point and optimal strategy. Based on the optimal strategy and optimal landing point, a phased control command sequence is generated, and the carrier is controlled based on the phased control command sequence. It is then determined whether the phase completion conditions are met or an abnormal state is triggered. If so, the decision-making process is switched to the next phase or a global replanning is triggered.

[0037] Thirdly, the present invention provides an electronic device including one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0038] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in any of the implementations of the first aspect.

[0039] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method as described in any of the implementations in the first aspect.

[0040] Compared with the prior art, the present invention has the following beneficial effects:

[0041] (1) The amphibious vessel unmanned driving decision-making method based on environmental perception and inference proposed in this invention introduces multimodal sensor data and deeply integrates multi-source sensor data such as vision, radar, and sonar. It utilizes a strategy generation module and a decision arbitration module based on a pre-trained large language model to achieve high-level semantic understanding, thereby significantly improving the perception and judgment capabilities of complex dynamic scenes.

[0042] (2) The amphibious vessel unmanned driving decision-making method based on environmental perception and inference proposed in this invention uses a world model to perform millisecond-level forward simulation, thereby realizing quantitative prediction of the ship's motion trajectory, attitude change and collision risk under different strategies. This makes the decision-making process have clear physical causal basis and foresight, improving the scientific nature and safety of the decision. In addition, it fully records the entire chain of information from environmental input, strategy generation, simulation and inference to final decision, forming an auditable decision log. This not only facilitates system debugging and optimization, but also meets the strict requirements for decision interpretability in high-reliability application scenarios such as emergency rescue.

[0043] (3) The amphibious vessel unmanned driving decision-making method based on environmental perception and inference proposed in this invention combines high-precision mechanical simulation and geological assessment during the water-land switching process to achieve fully automatic and adaptive landing / launching decision-making and control, which significantly improves the scenario adaptability and operational continuity of the amphibious system. Through the generalization understanding ability of the large model and the embedded modeling of physical laws by the world model, it can maintain stable decision-making performance in dynamic, partially observable and even conflict information environments, and has strong online learning and adaptability. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is a flowchart illustrating an embodiment of the unmanned driving decision-making method for amphibious vessels based on environmental perception and inference, according to an embodiment of this application.

[0046] Figure 2 This is a schematic diagram of the multi-sensor fusion and alignment process of the amphibious vessel unmanned driving decision-making method based on environmental perception and inference, which is an embodiment of this application.

[0047] Figure 3 This is a flowchart illustrating the "causal reasoning-scenario inference" decision-making mechanism based on a world model for the unmanned amphibious vessel decision-making method based on environmental perception and inference, which is an embodiment of this application.

[0048] Figure 4 This is a schematic diagram illustrating the process of generating control commands during the land-sea mode switching of the amphibious vessel unmanned driving decision-making method based on environmental perception and inference, which is an embodiment of this application.

[0049] Figure 5 This is a schematic diagram of an unmanned amphibious vessel decision-making device based on environmental perception and inference, as an embodiment of this application.

[0050] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0052] Figure 1 An embodiment of this application illustrates an unmanned driving decision-making method for amphibious vessels based on environmental perception and inference, comprising the following steps:

[0053] S1. Construct and train an unmanned decision-making model for amphibious ships to obtain a trained unmanned decision-making model for amphibious ships. The unmanned decision-making model for amphibious ships includes a world model and a policy generation module and a decision arbitration module based on a pre-trained large language model.

[0054] Specifically, the embodiments of this application employ a strategy generation module and a decision arbitration module based on a pre-trained large language model, combined with a world model, to construct an unmanned decision-making model for amphibious vessels. Further overall training and fine-tuning are then performed to obtain a trained unmanned decision-making model for amphibious vessels.

[0055] S2 acquires the multimodal sensor data collected at the current moment and performs preprocessing and analysis to obtain carrier state information, dynamic obstacle information and environmental feature parameters; encapsulates the carrier state information, dynamic obstacle information and environmental feature parameters with a timestamp into a unified environmental representation.

[0056] In a specific embodiment, the multimodal sensor data includes data collected by lidar, millimeter-wave radar, vision camera, IMU / GPS integrated navigation unit, sonar and environmental sensors. The IMU / GPS integrated navigation unit includes GPS, IMU and wheel speed meter. The preprocessing process includes filtering and noise reduction, distortion correction and spatiotemporal alignment to obtain preprocessed data corresponding to each sensor.

[0057] refer to Figure 2 The data analysis process is as follows:

[0058] Extended Kalman filtering is used to fuse preprocessed data from GPS, IMU, and wheel speedometer to calculate the latitude, longitude, and altitude of the vessel in the world coordinate system and convert them into three-dimensional position coordinates, while outputting a three-dimensional velocity vector. Complementary filtering is used to fuse acceleration and angular velocity data from the preprocessed data corresponding to the IMU to calculate the roll angle φ and pitch angle θ, and the heading ψ is obtained by integrating the angular velocity. The vessel's state information includes three-dimensional position coordinates, three-dimensional velocity vector, roll angle φ, pitch angle θ, and heading ψ.

[0059] Point cloud data corresponding to LiDAR or millimeter-wave radar is input into a 3D target detection network to generate 3D target detection results; visual images corresponding to visual cameras are input into a 2D target detection model to obtain 2D target detection results; the 2D target detection results are projected onto 3D space through the camera model and fused with the 3D target detection results to obtain the target position, size and category;

[0060] A multi-target tracking algorithm is used to correlate each target across frames to obtain the tracking trajectory, which is then combined with the target's position, size, and category to form the target's observation state.

[0061] Particle filtering is used to smooth the target observation state over time and predict the future state to obtain dynamic obstacle information, which includes the target position, size, velocity and heading angle.

[0062] The visual image corresponding to the visual camera is input into the semantic segmentation model to obtain the semantic segmentation result. The point cloud data corresponding to the LiDAR is processed by ground segmentation and meshing to construct a digital elevation model. The semantic segmentation result is fused with the pose of the LiDAR to obtain a three-dimensional semantic point cloud. The digital elevation model is aligned with the three-dimensional semantic point cloud space to obtain a three-dimensional semantic map. The environmental feature parameters include the three-dimensional semantic map and the digital elevation model.

[0063] Specifically, in the embodiments of this application, the multimodal sensor data used are asynchronously acquired raw data at their respective frequencies by the lidar, millimeter-wave radar, visual camera, IMU / GPS integrated navigation unit, sonar, and environmental sensors mounted on the amphibious vessel. All raw data are timestamped under a unified time base and undergo preprocessing processes such as filtering, noise reduction, and distortion correction to form a standardized data stream. The various standardized data streams are aligned in the spatiotemporal dimension, with the time synchronization error controlled within 1 ms, providing a consistent temporal basis for subsequent fusion and modeling. The amphibious vessel in the embodiments of this application is used as the carrier for analysis, and the analysis process is as follows:

[0064] First, an extended Kalman filter is used to perform multi-source fusion on the preprocessed data corresponding to GPS, IMU, and wheel speedometer in the IMU / GPS integrated navigation unit. This process calculates the high-precision latitude, longitude, and altitude information of the vehicle in the world coordinate system and converts it into local Cartesian coordinates [x, y, z]. Simultaneously, the three-dimensional velocity vector of the vehicle in the world coordinate system is output. , , In terms of attitude angle estimation, the roll angle φ and pitch angle θ are calculated by using complementary filtering to fuse the preprocessed data corresponding to the accelerometer and gyroscope. The bow direction ψ is obtained by integrating the angular velocity collected by the gyroscope, forming a complete carrier state vector.

[0065] Secondly, the point cloud data corresponding to LiDAR or millimeter-wave radar is processed by the PointPillars network to generate 3D target bounding boxes, thus obtaining 3D target detection results. The visual image, after detection by the YOLO model, is projected into 3D space through the camera model and combined with the 3D target detection results to obtain the target's position, size, and category. DeepSORT is used to perform cross-frame association and trajectory detection on each target, assigning a unique ID to each trajectory to obtain the tracking trajectory, which is then combined with the target's position, size, and category to form the target observation state. Particle filtering is used to smooth the noisy and abrupt target observation states (such as position and velocity) generated by multimodal detection and cross-frame association, and to predict future states, thereby outputting more stable and continuous dynamic obstacle information, including its target position, size, velocity, and heading angle. Therefore, fusing multimodal detection results can obtain more stable target position, size, and category information.

[0066] Then, DeepLab or SegFormer are used to perform semantic segmentation on the visual images, and a 3D semantic map is constructed by combining visual SLAM or laser SLAM. Ground segmentation and meshing are performed on the point cloud data to construct a digital elevation model (DEM). The semantic segmentation results and the DEM are used to generate a 3D semantic map. The DEM and the 3D semantic map constitute environmental feature parameters.

[0067] The vehicle's state information, dynamic obstacle information, and environmental feature parameters are encapsulated into a unified environmental representation using a unified timestamp. The vehicle state information includes three-dimensional position coordinates, a three-dimensional velocity vector, and attitude angles, including roll angle φ, pitch angle θ, and heading ψ. The dynamic obstacle information includes the obstacle's unique trajectory ID, target category, three-dimensional position and size, velocity, and heading angle. The environmental feature parameters include a three-dimensional semantic map and a digital elevation model. This information serves as the standard input to the amphibious vessel's unmanned decision-making model. An example is shown below:

[0068] {

[0069] "timestamp": 1730100605123,

[0070] “ego_state”: {

[0071] "position": {"x": 100.5, "y": 200.3, "z": 0.8},

[0072] "velocity": {"vx": 2.1, "vy": 0.5, "vz": 0.0},

[0073] “attitude”: {“roll”: 1.5, “pitch”: 0.3, “yaw”: 45.2}

[0074] },

[0075] “dynamic_obstacles”: [

[0076] {"id": 101, "position": {"x": 150.0, "y": 190.0, "z":0.0}, "velocity": 3.0, "heading": 30.0, "class": "fishing_boat"},

[0077] {"id": 102, "position": {"x": 80.0, "y": 220.0, "z":0.5}, "velocity": 1.5, "heading": 120.0, "class": "buoy"}

[0078] ],

[0079] "static_environment": {

[0080] "local_map_slice": {

[0081] "center": {"x": 100.5, "y": 200.3}, / / Center of the slice (i.e., the location of the carrier)

[0082] "size": {"width": 100.0, "height": 100.0}, / / Slice range (meters)

[0083] "resolution": 0.5, / / Grid resolution (meters per grid)

[0084] "layers": { / / Contains multiple information layers, corresponding to different outputs from step four.

[0085] "semantic": {

[0086] "type": "occupancy_grid",

[0087] "data": "BASE64_ENCODED_GRID_DATA",

[0088] "classes": {"0": "water", "1": "shore", "2": "rock", "3": "buoy"}

[0089] },

[0090] "elevation": { / / Layer 2: Digital Elevation Model (DEM)

[0091] "type": "height_grid",

[0092] "data": "BASE64_ENCODED_GRID_DATA"

[0093] },

[0094] }

[0095] In a specific embodiment, the physical state of the ship maintained in the world model is determined using the following steps:

[0096] The current roll angle φ and bow heading ψ are combined to form the nominal physical state at the current moment. ;

[0097] The world model extrapolates the predicted physical state of the ship at the current moment based on the ship's physical state and dynamic model at the previous moment. The system calculates the difference between the nominal physical state and the predicted physical state at the current moment. If the difference exceeds a threshold, a state correction process is triggered. In the state correction process, the carrier state information is used as an observation constraint. A factor graph optimization algorithm is adopted, combined with dynamic model constraints, to smooth and backtrack the physical state within the time window corresponding to the current moment of the world model, thereby obtaining the optimized physical state at the current moment. The optimized physical state at the current moment is taken as the physical state of the ship at the current moment. If the difference does not exceed the threshold, the predicted physical state at the current moment is taken as the physical state of the ship at the current moment.

[0098] Specifically, the world model maintains a dynamic physical state. Considering that the roll angle ϕ is the core indicator for evaluating ship stability, and the heading ψ is the fundamental variable controlling the navigation direction and trajectory, these two together constitute the most critical state inputs for the world model to perform physical simulations, risk predictions, and attitude adjustments during water-to-land transitions. The embodiments of this application combine the roll angle φ and heading ψ extracted from the carrier state information to form a nominal physical state. Meanwhile, the world model extrapolates the predicted angle for the current moment based on the ship's physical state and dynamics model from the previous moment. and The predicted physical state formed Calculate the difference between the two: Δ= When the difference Δ exceeds a preset threshold, a state correction process is triggered. During correction, the position, velocity, and attitude angles in the carrier's state information are used as observation constraints. A factor graph optimization algorithm, combined with dynamic model constraints, is employed to smooth and backtrack the physical state within a time window within the world model, ensuring that the world model's predictions are consistent with the actual physical state. The optimized physical state will serve as a more reliable initial condition for subsequent simulations and predictions.

[0099] S3. Input the unified environment representation, the physical state of the ship maintained in the world model, and the mission command text into the trained amphibious unmanned decision-making model. First, the strategy generation module generates a strategy framework, which includes several candidate strategies. Then, the world model is used to perform dynamic simulation on each candidate strategy to generate a quantitative evaluation report. The quantitative evaluation report is input into the decision arbitration module, and a preset safety rule base and mission objective weights are loaded to generate a recommended strategy and drive according to the recommended strategy.

[0100] In a specific embodiment, the strategy framework includes action type, main objective, constraints and candidate strategies; the quantitative evaluation report includes the predicted trajectory, minimum encounter distance, maximum roll angle, estimated arrival time, energy consumption and rule risk score for each candidate strategy.

[0101] For details, please refer to Figure 3 The system inputs the ship's physical state and mission command text maintained in the unified environment representation and world model into the trained amphibious vessel unmanned decision-making model. After passing through the policy generation module, the structured data undergoes semantic understanding and summary generation, such as identifying potential behavioral trends of dynamic obstacles. Subsequently, core decision points (such as avoidance constraints, stability limits, and energy consumption targets) are extracted to generate a structured policy framework, for example:

[0102] {

[0103] "action_type": "dynamic_obstacle_avoidance",

[0104] "primary_target": "charging_dock",

[0105] "constraints": ["min_clearance > 15m", "max_roll < 10deg"],

[0106] "candidate_strategies": [

[0107] {"id": "A", "params": {"delta_heading_deg": 5, "throttle_percent": -20}},

[0108] {"id": "B", "params": {"delta_heading_deg": -10, "throttle_percent": 0}} ]

[0110] }

[0111] The world model is used to dynamically simulate each candidate strategy, generating a complete predicted trajectory and outputting a quantitative evaluation report, including minimum encounter distance, maximum roll angle, estimated arrival time, energy consumption, and rule risk score. For example:

[0112] The quantitative assessment report for Strategy A is as follows: {“Strategy ID”: “A”, “Predicted Trajectory”: [Path Point List], “Minimum Encounter Distance”: 18.5m, “Maximum Roll Angle”: 12.3°, “Estimated Arrival Time”: 215s, “Energy Consumption Estimation”: 150kJ, “Rule Risk Score”: 0.15};

[0113] The quantitative assessment report for Strategy B is as follows: {“Strategy ID”: “B”, “Predicted Trajectory”: [Path Point List], “Minimum Encounter Distance”: 25.2m, “Maximum Roll Angle”: 8.1°, “Estimated Arrival Time”: 240s, “Estimated Energy Consumption”: 135kJ, “Rule Risk Score”: 0.05}.

[0114] The decision-making arbitration module receives quantitative evaluation reports of all candidate strategies and loads a pre-set safety rule base (such as "risk score > 0.1 should be avoided", "roll angle > 10° may affect operation") and task objective weights (such as "timeliness weight 0.3, safety weight 0.5, energy consumption weight 0.2"). A large language model acts as the decision arbitrator, weighting and evaluating each indicator. For example, strategy A is fast but has high roll and risk; strategy B is more stable and safer but slightly slower. Then, the large language model outputs the final selection and detailed reasons, such as: "Strategy B is selected. Reason: Although the expected arrival time increases by 25 seconds, its minimum encounter distance (25.2m) is far higher than the safety threshold, the maximum roll angle (8.1°) ensures the stability of subsequent charging docking, and the overall risk score is the lowest (0.05). This decision prioritizes safety and operational quality."

[0115] S4. During the water-land transition, hydrological analysis is performed based on environmental characteristic parameters to generate a list of candidate landing points. Each candidate landing point and recommended strategy in the list is evaluated and mechanically simulated to determine the optimal landing point and optimal strategy. Based on the optimal strategy and optimal landing point, a phased control command sequence is generated. The carrier is controlled based on the phased control command sequence, and it is determined whether the phase completion conditions are met or an abnormal state is triggered. If so, the decision-making process of the next phase is switched or a global replanning is triggered.

[0116] In a specific embodiment, during the land-water transition process, hydrological analysis is performed based on environmental characteristic parameters to generate a candidate landing point list. Each candidate landing point and recommended strategy in the list are evaluated and subjected to mechanical simulation to determine the optimal landing point and optimal strategy. Specifically, this includes:

[0117] Key terrain features are calculated based on the digital elevation model. These key terrain features include slope changes, protruding obstacles, and depressions. Slope changes are directly obtained by calculating the normal vector of grid vertices or the elevation difference between adjacent points. Protruding obstacles are identified by detecting positive elevation changes in a local area relative to the surrounding terrain that exceed an elevation threshold. Depressions are identified by detecting negative elevation changes in a local area that exceed an elevation threshold.

[0118] A dual-branch feature fusion-based geological hardness classification model was constructed and trained to obtain the trained geological hardness classification model. This model includes an image branch, a vibration branch, a fusion module, and a classifier. The image branch uses a pre-trained ResNet network, and the vibration branch uses a temporal Transformer encoder. Visual images and temporal data from chassis vibration sensors were input into the trained geological hardness classification model. The visual images were processed through the image branch to extract high-dimensional texture, color, and structural features. The temporal data from the chassis vibration sensors were processed through the vibration branch to extract features related to bottoming impact and resonant frequency. The high-dimensional texture, color, and structural features, as well as the features related to bottoming impact and resonant frequency, were input into the fusion module for fusion to obtain fused features. These fused features were then processed by the classifier to obtain the probability of each geological hardness category. The geological hardness was determined based on the probability of each geological hardness category.

[0119] Hydrological information is obtained by assessing nearshore water depth distribution, geological conditions, and wave impact intensity using data collected by sonar. Environmental characteristic parameters are comprehensively evaluated by integrating key topographic features, geological hardness, and hydrological information to generate a list of candidate landing points. The data for each candidate landing point includes spatial coordinates, suggested shore angle, and preliminary velocity.

[0120] In the mechanical simulation process, a multi-domain coupled dynamic model based on the fluid domain, structural domain, and ground contact domain is constructed. The fluid domain is used to describe the hydrodynamic action, the structural domain is used to describe the rigid-flexible coupling response of the carrier, and the ground contact domain is used to describe the nonlinear contact mechanical behavior between the tire or track and the terrain. The data and environmental information of each candidate landing point are input into the multi-domain coupled dynamic model to calculate the mechanical indices. Based on the mechanical indices, the optimal landing point is selected, and the recommended strategy is optimized to obtain the optimal strategy.

[0121] Specifically, key terrain features such as slope changes, protruding obstacles, and concave areas are calculated based on the digital elevation model. Slope is directly obtained by calculating the normal vector of grid vertices or the elevation difference between adjacent points; protruding obstacles are identified by setting elevation thresholds and detecting significant positive elevation changes in local areas relative to the surrounding terrain; and concave areas are identified by detecting negative elevation changes in local areas.

[0122] Further, the results of close-range visual texture analysis can be combined with the bottom-touching feedback from chassis vibration sensors in shallow water areas. A trained geological hardness classification model is used to estimate geological bearing capacity in real time, outputting the probability distribution of soft, medium, and hard levels. The core of this trained geological hardness classification model is a multi-branch feature fusion neural network. First, a pre-trained ResNet is used as the backbone network in the image branch, responsible for extracting high-dimensional texture, color, and structural features from close-range visual images. Then, a temporal Transformer encoder is used in the vibration branch to extract features related to bottom-touching impact and resonant frequency from the temporal data of the chassis vibration sensors. Finally, the high-dimensional feature vectors extracted from the two branches are concatenated or fused using an attention mechanism. The fused features undergo dimensionality reduction and nonlinear transformation through a fully connected layer, and finally, a Softmax layer outputs the probability distribution of the three geological hardness categories. Sonar data is used to assess nearshore water depth distribution, bottom conditions, and wave impact intensity, thereby completing the hydrological condition analysis. Integrating multi-dimensional information from topography, geology, and hydrology, unsafe areas are eliminated, and a candidate landing point list is generated based on a comprehensive score. Each candidate landing point includes spatial coordinates, a suggested landing angle, and preliminary velocity parameters, providing input for subsequent high-precision simulations.

[0123] Unsafe areas are defined by a series of hard safety thresholds; if any one of them is not met, the area is removed.

[0124] 1. Unsafe terrain: The slope exceeds the vehicle's maximum climbing angle, or there are insurmountable obstacles / deep pits.

[0125] 2. Geological insecurity: The probability of "soft" geology output by the geological classification model exceeds the threshold, indicating insufficient pressure bearing capacity.

[0126] 3. Hydrological insecurity: Insufficient water depth measured by sonar, soft bottom, or wave impact exceeding the stability threshold.

[0127] First, the aforementioned hard conditions are used to filter environmental characteristic parameters by region, eliminating all unsafe areas. Then, for the remaining safe areas, a comprehensive score is calculated based on indicators such as slope (the smaller the better), geological bearing capacity (the harder the better), and water surface stability, generating a list of candidate landing points sorted by score.

[0128] Based on this, a closed-loop decision-making mechanism driven by high-precision mechanical simulation is introduced to achieve closed-loop optimization control of "environment modeling - candidate generation - simulation evaluation - strategy optimization - execution feedback" throughout the entire process of water-land switching.

[0129] Specifically, the high-precision mechanical simulation process includes: constructing a multi-domain coupled dynamic model covering the "fluid domain, structural domain, and ground contact domain," where the fluid domain describes hydrodynamic effects (including buoyancy, drag, and wave impact), the structural domain describes the rigid-flexible coupling response of the carrier, and the ground contact domain describes the nonlinear contact mechanical behavior between the tires / tracks and the terrain; during the simulation, the landing angle, velocity parameters, chassis configuration, and environmental information (slope, water depth, seabed, etc.) corresponding to the candidate landing point are used as inputs to the multi-domain coupled dynamic model to calculate key mechanical indicators such as peak impact force, attitude stabilization time, slip ratio, and subsidence; based on the mechanical indicators, a multi-objective evaluation function is constructed to comprehensively and quantitatively evaluate safety, stability, and passability, thereby selecting the optimal landing point, and optimizing key parameters in the recommended strategy (such as landing speed, suspension stiffness, and attitude control gain) to obtain the optimal strategy.

[0130] In a specific embodiment, a phased control instruction sequence is generated based on the optimal strategy and the optimal landing point, specifically including:

[0131] The optimal strategy is analyzed as a moving target, and the planned trajectory is obtained by combining the dynamic constraints of the optimal landing point;

[0132] Based on the geometric and temporal characteristics of the planned trajectory and combined with the task semantics, the planned trajectory is divided into several stages. A core control objective and a corresponding control mode are assigned to each stage, resulting in a corresponding phased control instruction sequence.

[0133] Specifically, based on the optimal landing point and optimal strategy, a phased control command sequence is generated and executed by the underlying controller. The specific steps are as follows:

[0134] 1. The optimal strategy (such as "dynamic obstacle avoidance followed by docking") is analyzed as a specific motion target. Based on the location of the optimal landing point, the environment, and the dynamic constraints of the carrier, a collision-free, time-parameterized, and smooth trajectory is planned. This trajectory defines a complete state sequence (changes in position, velocity, and attitude over time) from the starting point to the ending point.

[0135] 2. Based on the geometric and temporal characteristics of the planned trajectory (such as curvature, velocity profile, and acceleration), and combined with the task semantics, the continuous planned trajectory is automatically divided into multiple logically continuous stages. Each stage is assigned a core control objective (such as "precise path tracking", "maintain constant speed", and "stable attitude") and a corresponding control mode (such as "trajectory tracking", "cruise control", and "position holding").

[0136] 3. Generate corresponding control commands for each segmented stage. These control commands are not low-level servo signals, but rather define the objectives and parameters of the control law for that stage, for example:

[0137] {“Control Mode”: “Track Tracking”, “Reference Track”: [Track Segment], “Maximum Lateral Error”: 0.1m};

[0138] {“Control Mode”: “Speed ​​Hold”, “Target Speed”: 1.5m / s, “Maximum Acceleration”: 0.3m / s²};

[0139] {“Control Mode”: “System Configuration”, “Action”: “Suspension Preload”, “Target Stiffness”: 80%}.

[0140] The phased control command sequence is handed over to the underlying controller for execution. Based on the control command of the current phase, the controller calculates and outputs the underlying drive signals (such as throttle, rudder angle, and torque) in real time. It continuously monitors the matching degree between the vehicle's state and the phase objective, and provides feedforward based on predictions from the world model. If the phase completion conditions are met (such as reaching the setpoint) or abnormal conditions are triggered (such as exceeding deviation limits or sudden environmental changes), it will automatically switch to the next phase command or trigger a global replanning.

[0141] refer to Figure 4 Example of a phased control instruction sequence:

[0142] Phase 1 (Precise Alignment on Water): Control Command: {"Target Point": [Optimal Landing Point Coordinates], "Desired Course": [Calculated Landing Angle], "Mode": "Fine Navigation"}. Simultaneously, the pre-command suspension system enters "Ready to Land" mode and adjusts to the preset stiffness.

[0143] Phase Two (Shallow Water Area Deceleration and Landing): When a certain distance is reached from the landing point, control commands are given: {"Speed ​​Curve", "Smoothly Decay to Touch-Off Speed", "Chassis Height": "Adjust to Preset Touch-Off Height"}.

[0144] Phase 3 (Active Ground Contact Impact Management): At the moment of tire contact with the ground, based on feedback from the force sensor, the active suspension control system executes a preset buffering algorithm (such as rapid depressurization and repressurization) within milliseconds to actively absorb impact energy.

[0145] Phase Four (Land Attitude Stabilization and Mode Switching): After confirming that the vehicle is completely stable on land, control commands are given: {"Power Source Switching": "Propeller -> Tire", "Navigation Map Switching": "Maritime Map to Land Map", "Suspension Mode": "Land Driving"}, to complete the fully automatic transition.

[0146] At the control execution level, the optimal strategy is parsed into a control objective with time-series constraints, and a continuous and feasible reference trajectory is generated by combining the spatial and dynamic constraints of the optimal landing point. Based on the geometric and dynamic characteristics of the reference trajectory, the task process is divided into stages, forming a phased control command sequence. During execution, the control system outputs the underlying control quantity using a combination of feedforward prediction and feedback correction, and evaluates the execution effect based on real-time sensing data. When a stage objective is detected to be achieved (such as position error convergence or attitude stabilization), the system automatically switches to the next stage. When an abnormal state is detected (including but not limited to attitude deviation exceeding limits, abnormal increase in impact force, or sudden change in geological bearing capacity), global replanning is triggered to re-evaluate candidate landing points and strategies to ensure the robustness and safety of the system in complex environments.

[0147] Further reference Figure 5 As an implementation of the methods shown in the above figures, this application provides an embodiment of an unmanned decision-making device for amphibious vessels based on environmental perception and inference. This device embodiment is similar to... Figure 1 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0148] This application provides an unmanned decision-making device for amphibious vessels based on environmental perception and simulation, including:

[0149] Model building module 1 is configured to build and train an amphibious unmanned decision-making model, resulting in a trained amphibious unmanned decision-making model. The amphibious unmanned decision-making model includes a world model and a policy generation module and a decision arbitration module based on a pre-trained large language model.

[0150] The multimodal data processing module 2 is configured to acquire the multimodal sensor data collected at the current moment and perform preprocessing and analysis to obtain carrier state information, dynamic obstacle information and environmental feature parameters; and encapsulate the carrier state information, dynamic obstacle information and environmental feature parameters with a timestamp into a unified environmental representation;

[0151] The strategy generation module 3 is configured to input the unified environment representation, the physical state of the ship maintained in the world model, and the mission instruction text into the trained amphibious vehicle unmanned decision-making model. First, the strategy generation module generates a strategy framework, which includes several candidate strategies. Then, the world model is used to perform dynamic simulation on each candidate strategy to generate a quantitative evaluation report. The quantitative evaluation report is then input into the decision arbitration module, and a preset safety rule base and mission objective weights are loaded to generate a recommended strategy and drive according to the recommended strategy.

[0152] Planning module 4 is configured to perform hydrological analysis based on environmental characteristic parameters during the water-land transition process, generate a list of candidate landing points, evaluate and perform mechanical simulation on each candidate landing point and recommended strategy in the candidate landing point list, determine the optimal landing point and optimal strategy, generate a phased control command sequence based on the optimal strategy and optimal landing point, control the carrier based on the phased control command sequence, and determine whether the phase completion conditions are met or an abnormal state is triggered. If so, switch to the decision-making process of the next phase or trigger global replanning.

[0153] Figure 6 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. For example... Figure 6 As shown, the electronic device of this embodiment includes a processor 601 and a memory 602; wherein the memory 602 is used to store computer execution instructions; and the processor 601 is used to execute the computer execution instructions stored in the memory to implement the various steps performed by the electronic device in the above embodiment. For details, please refer to the relevant descriptions in the foregoing method embodiments.

[0154] Alternatively, the memory 602 can be either standalone or integrated with the processor 601.

[0155] When the memory 602 is set up independently, the electronic device also includes a bus 603 for connecting the memory 602 and the processor 601.

[0156] This invention also provides a computer storage medium storing computer execution instructions, which, when executed by processor 601, implement the above method.

[0157] This invention also provides a computer program product, including a computer program that, when executed by a processor 601, implements the above-described method.

[0158] In the embodiments provided by this invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or modules, and may be electrical, mechanical, or other forms.

[0159] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to implement the solution of this embodiment according to actual needs.

[0160] Furthermore, the functional modules in the various embodiments of this invention can be integrated into one processing unit, or each module can exist physically separately, or two or more modules can be integrated into one unit. The unit formed by the above modules can be implemented in hardware or in the form of hardware plus software functional units.

[0161] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor 601 to execute some steps of the methods of the various embodiments of this application.

[0162] It should be understood that the processor 601 described above can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor, or the processor 601 can be any conventional processor 601. The steps of the method disclosed in this invention can be directly manifested as the hardware processor 601 executing the steps, or as a combination of hardware and software modules within the processor 601 executing the steps.

[0163] The memory 602 may include high-speed RAM memory, and may also include non-volatile memory NVM, such as at least one disk storage device, and may also be a USB flash drive, portable hard drive, read-only memory, disk or optical disc, etc.

[0164] Bus 603 can be an Industry Standard Architecture (ISA), a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Bus 603 can be divided into address bus, data bus, control bus, etc. For ease of illustration, the bus 603 in the accompanying drawings of this application is not limited to only one bus 603 or one type of bus 603.

[0165] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0166] An exemplary storage medium is coupled to a processor 601, enabling the processor 601 to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor 601. The processor 601 and the storage medium can reside in an application-specific integrated circuit (ASIC). Alternatively, the processor 601 and the storage medium can exist as discrete components in an electronic device or a host device.

[0167] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An amphibious vehicle autonomous decision-making method based on environment perception deduction, characterized in that, Includes the following steps: An unmanned decision-making model for amphibious ships is constructed and trained to obtain a trained unmanned decision-making model for amphibious ships. The unmanned decision-making model for amphibious ships includes a world model and a policy generation module and a decision arbitration module based on a pre-trained large language model. The system acquires multimodal sensor data collected at the current moment, performs preprocessing and analysis to obtain carrier state information, dynamic obstacle information, and environmental feature parameters; and encapsulates the carrier state information, dynamic obstacle information, and environmental feature parameters with a timestamp into a unified environmental representation. The unified environment representation, the ship's physical state maintained in the world model, and the mission command text are input into the trained amphibious unmanned decision-making model. First, the strategy generation module generates a strategy framework, which includes several candidate strategies. Then, the world model is used to dynamically simulate each candidate strategy to generate a quantitative evaluation report. The quantitative evaluation report is input into the decision arbitration module, and a preset safety rule base and mission objective weights are loaded to generate a recommended strategy and drive according to the recommended strategy. During the water-land transition, hydrological analysis is performed based on the environmental characteristic parameters to generate a candidate landing point list. Each candidate landing point and recommended strategy in the candidate landing point list is evaluated and mechanically simulated to determine the optimal landing point and optimal strategy. A phased control command sequence is generated based on the optimal strategy and optimal landing point. The carrier is controlled based on the phased control command sequence, and it is determined whether the phase completion conditions are met or an abnormal state is triggered. If so, the decision-making process of the next phase is switched or a global replanning is triggered.

2. The method of claim 1, wherein, The multimodal sensor data includes data collected by lidar, millimeter-wave radar, vision camera, IMU / GPS integrated navigation unit, sonar and environmental sensors. The IMU / GPS integrated navigation unit includes GPS, IMU and wheel speed meter. The preprocessing process includes filtering and noise reduction, distortion correction and spatiotemporal alignment to obtain preprocessed data corresponding to each sensor. The analysis process is as follows: Extended Kalman filtering is used to fuse preprocessed data from GPS, IMU, and wheel speedometer to calculate the latitude, longitude, and altitude of the vehicle in the world coordinate system and convert them into three-dimensional position coordinates, while simultaneously outputting a three-dimensional velocity vector. Complementary filtering is used to fuse acceleration and angular velocity data from the preprocessed data corresponding to the IMU to calculate the roll angle φ and pitch angle θ, and the heading ψ is obtained by integrating the angular velocity. The vehicle state information includes the three-dimensional position coordinates, three-dimensional velocity vector, roll angle φ, pitch angle θ, and heading ψ. The point cloud data corresponding to the lidar or millimeter-wave radar is input into the three-dimensional target detection network to generate a three-dimensional target detection result; the visual image corresponding to the visual camera is input into the two-dimensional target detection model to obtain a two-dimensional target detection result; the two-dimensional target detection result is projected onto the three-dimensional space through the camera model and fused with the three-dimensional target detection result to obtain the target position, size and category; A multi-target tracking algorithm is used to correlate each target across frames to obtain the tracking trajectory, which is then combined with the target's position, size, and category to form the target's observation state. Particle filtering is used to smooth the target observation state over time and predict future states to obtain dynamic obstacle information, which includes the target position, size, speed and heading angle. The visual image corresponding to the visual camera is input into the semantic segmentation model to obtain the semantic segmentation result. The point cloud data corresponding to the lidar is subjected to ground segmentation and meshing processing to construct a digital elevation model. The semantic segmentation result is fused with the lidar pose to obtain a three-dimensional semantic point cloud. The digital elevation model is spatially aligned with the three-dimensional semantic point cloud to obtain a three-dimensional semantic map. The environmental feature parameters include the three-dimensional semantic map and the digital elevation model.

3. The method of claim 2, wherein, The physical state of the ship maintained in the world model is determined using the following steps: combining the current instant of the roll angle φ and the ship heading ψ to form a nominal physical state at the current instant ; The world model extrapolates the predicted physical state of the ship at the current moment based on the ship's physical state and dynamic model at the previous moment. The difference between the nominal physical state and the predicted physical state at the current moment is calculated. If the difference exceeds a threshold, the state correction process is triggered. In the state correction process, the carrier state information is used as the observation constraint. The factor graph optimization algorithm is used in combination with the dynamic model constraint to smooth and backtrack the physical state within the time window corresponding to the current moment of the world model, so as to obtain the optimized physical state at the current moment. The optimized physical state at the current moment is used as the physical state of the ship at the current moment. If the difference does not exceed the threshold, the predicted physical state at the current moment is taken as the physical state of the ship at the current moment.

4. The unmanned driving decision-making method for amphibious vessels based on environmental perception and inference according to claim 1, characterized in that, The strategy framework includes action type, main objective, constraints and candidate strategies; the quantitative evaluation report includes the predicted trajectory, minimum encounter distance, maximum roll angle, estimated arrival time, energy consumption and rule risk score for each candidate strategy.

5. The unmanned driving decision-making method for amphibious vessels based on environmental perception and inference according to claim 2, characterized in that, During the land-water transition, hydrological analysis is performed based on the environmental characteristic parameters to generate a candidate landing point list. Each candidate landing point and recommended strategy in the list are evaluated and subjected to mechanical simulation to determine the optimal landing point and optimal strategy. Specifically, this includes: Key terrain features are calculated based on the digital elevation model. These key terrain features include slope changes, protruding obstacles, and recessed areas. The slope changes are directly obtained by calculating the normal vector of the grid vertex or the elevation difference between adjacent points. The protruding obstacles are identified by detecting positive elevation changes in a local area relative to the surrounding terrain that exceed an elevation threshold. The recessed areas are identified by detecting negative elevation changes in a local area that exceed an elevation threshold. A dual-branch feature fusion-based geological hardness classification model is constructed and trained to obtain a trained geological hardness classification model. This model includes an image branch, a vibration branch, a fusion module, and a classifier. The image branch uses a pre-trained ResNet network, and the vibration branch uses a temporal Transformer encoder. Visual images and temporal data corresponding to chassis vibration sensors are input into the trained geological hardness classification model. The visual images are processed through the image branch to extract high-dimensional texture features, color features, and structural features. The temporal data corresponding to the chassis vibration sensors are processed through the vibration branch to extract features related to bottoming impact and resonant frequency. The high-dimensional texture features, color features, structural features, and features related to bottoming impact and resonant frequency are input into the fusion module for fusion to obtain fused features. These fused features are then processed by the classifier to obtain the probability of each geological hardness category. The geological hardness is determined based on the probability of each geological hardness category. The sonar data is used to assess the nearshore water depth distribution, geological conditions and wave impact intensity to obtain hydrological information; the key topographic features, geological hardness and hydrological information are combined to comprehensively evaluate the environmental characteristic parameters and generate a list of candidate landing points. The data of each candidate landing point includes spatial coordinates, suggested landing angle and initial velocity. In the mechanical simulation process, a multi-domain coupled dynamic model based on the fluid domain, structural domain, and ground contact domain is constructed. The fluid domain is used to describe the hydrodynamic action, the structural domain is used to describe the rigid-flexible coupling response of the carrier, and the ground contact domain is used to describe the nonlinear contact mechanical behavior between the tire or track and the terrain. The data and environmental information of each candidate landing point are input into the multi-domain coupled dynamic model to calculate the mechanical indices. Based on the mechanical indices, the optimal landing point is selected, and the recommended strategy is optimized to obtain the optimal strategy.

6. The unmanned driving decision-making method for amphibious vessels based on environmental perception and inference according to claim 1, characterized in that, Based on the optimal strategy and optimal landing point, a phased control instruction sequence is generated, specifically including: The optimal strategy is analyzed as a moving target, and the planned trajectory is obtained by combining the dynamic constraints of the optimal landing point; Based on the geometric and temporal characteristics of the planned trajectory and combined with the task semantics, the planned trajectory is divided into several stages, and a core control objective and a corresponding control mode are assigned to each stage to obtain the corresponding staged control instruction sequence.

7. An unmanned decision-making device for amphibious vessels based on environmental perception and inference, characterized in that, include: The model building module is configured to build and train an unmanned decision-making model for amphibious vessels, resulting in a trained unmanned decision-making model for amphibious vessels. The unmanned decision-making model for amphibious vessels includes a world model and a policy generation module and a decision arbitration module based on a pre-trained large language model. The multimodal data processing module is configured to acquire multimodal sensor data collected at the current moment, perform preprocessing and analysis to obtain carrier state information, dynamic obstacle information and environmental feature parameters; and encapsulate the carrier state information, dynamic obstacle information and environmental feature parameters with a timestamp into a unified environmental representation; The strategy generation module is configured to input the unified environment representation, the ship's physical state maintained in the world model, and the mission instruction text into the trained amphibious unmanned decision-making model. First, the strategy generation module generates a strategy framework, which includes several candidate strategies. Then, it performs dynamic simulation on each candidate strategy using the world model to generate a quantitative evaluation report. Finally, it inputs the quantitative evaluation report into the decision arbitration module, loads a preset safety rule base and mission objective weights, generates a recommended strategy, and drives according to the recommended strategy. The planning module is configured to perform hydrological analysis based on the environmental characteristic parameters during the water-land transition process, generate a list of candidate landing points, evaluate and perform mechanical simulation on each candidate landing point and recommended strategy in the candidate landing point list, and determine the optimal landing point and optimal strategy; generate a phased control command sequence based on the optimal strategy and optimal landing point, control the carrier based on the phased control command sequence, and determine whether the phase completion conditions are met or an abnormal state is triggered. If so, switch to the decision-making process of the next phase or trigger global replanning.

8. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-6.

Images