An event-driven multi-modal adaptive fusion decision method for autonomous navigation of unmanned ships

Abstract

The application discloses an event-driven multi-modal adaptive fusion decision method for autonomous navigation of unmanned ships, and belongs to the technical field of intelligent decision of unmanned autonomous systems. The method aims at the problem that the visual environment information and the ship state information are heterogeneous during the navigation of the unmanned ship, which leads to the difficulty in multi-source information fusion. The visual information and the ship state information are acquired and synchronously processed. The continuous environment information and the motion state information are converted into event features through a visual event encoding path and a ship state event encoding path respectively. The event features of different sources are represented based on a unified event time axis. Adaptive fusion parameters are generated according to the visual event change and the ship state change, and multi-modal feature dynamic fusion is realized. The fusion features are input into an event response unit, a navigation strategy is determined, and the navigation strategy is sent to an unmanned ship execution system. The application realizes event expression, adaptive fusion and autonomous navigation decision of multi-source heterogeneous information of the unmanned ship.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology for unmanned autonomous systems, specifically to an event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels. This method addresses the different change frequencies and dynamic characteristics of visual environment information and hull motion state information during unmanned vessel navigation. By constructing an event-triggered information encoding and state accumulation decision-making mechanism, it achieves sparse representation, adaptive fusion, and rapid policy response of heterogeneous navigation information. Background Technology

[0002] With the development of unmanned systems, intelligent sensing, and autonomous navigation technologies, unmanned surface vessels (USVs) have been widely applied in fields such as waterway inspection, environmental monitoring, target search, and intelligent transportation. During autonomous navigation, USVs need to select appropriate navigation strategies in real time based on changes in the surrounding water environment and their own motion status. Visual sensors acquire information about the external environment, such as shorelines, obstacles, water surface textures, and changes in lighting; inertial measurement units, positioning devices, velocity sensors, and control status information reflect the vessel's attitude, speed, inertial changes, and current operating status. Because environmental perception information and vessel status information differ in data format, sampling methods, and dynamic change patterns, effectively fusing multi-dimensional heterogeneous information to form reliable navigation decisions is a crucial challenge for autonomous navigation of USVs.

[0003] Existing autonomous decision-making methods for unmanned surface vessels (USVs) typically employ a deep learning-based perception and decision-making framework. This framework uses convolutional neural networks to extract environmental features from visual images and recurrent neural networks, attention networks, or other temporal models to process hull state information. During multimodal information fusion, some methods directly concatenate or jointly input features from different sources. However, due to differences in feature scale, temporal variation patterns, and information representation between visual and physical state information, intermodal interference can easily occur, reducing the decision-making model's adaptability to complex navigation environments.

[0004] Furthermore, some existing methods enhance the correlation between different information by adding cross-modal feature interaction modules. However, multi-level feature interaction processes typically require significant computational and storage resources, which is not conducive to real-time deployment on edge computing platforms such as unmanned surface vessels (USVs). Meanwhile, traditional continuous value neural networks usually perform continuous calculations on historical information based on fixed time windows before outputting corresponding navigation strategies. When USVs encounter sudden obstacles, rapid environmental changes, or alterations in motion, the decision-making model requires a lengthy computational process to update its strategy, potentially leading to delayed navigation decision responses.

[0005] On the other hand, existing multimodal decision-making methods typically employ fixed-weight fusion or probability-based result fusion to statically assign the importance of different information sources. However, in actual navigation, the reliability of visual information may be affected by factors such as illumination, occlusion, and water surface reflection, while the importance of hull state information also changes with the motion state. Fixed fusion methods struggle to dynamically adjust the contribution ratio of different modal information according to the current navigation state.

[0006] Therefore, in order to address the problems of inconsistent expression of multi-source heterogeneous information, interference in the fusion process, insufficient decision response speed, and lack of dynamic adaptability of fusion strategies during the autonomous navigation of unmanned vessels, a multimodal decision-making method is needed that can maintain the independent feature expression of different modal information, perform adaptive fusion according to the current navigation state, and use an event-driven mechanism to quickly complete the navigation strategy selection, so as to improve the accuracy, real-time performance, and adaptability of autonomous navigation decisions of unmanned vessels in complex environments. Summary of the Invention

[0007] The technical problem that this invention aims to solve.

[0008] To address the problems in existing autonomous navigation decision-making processes for unmanned vessels, such as the difficulty in effectively fusing multi-source heterogeneous information, the easy interference between different modal features, the insufficient response speed of traditional continuous computing models, and the difficulty of adapting fixed fusion strategies to complex navigation state changes, this invention provides an event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels.

[0009] This invention constructs a visual information event encoding path and a hull state event encoding path to convert continuously changing environmental perception information and motion state information into time-correlated event features; further, it generates dynamic fusion parameters based on the current navigation state to achieve adaptive adjustment of the contribution level of different modal information; at the same time, it uses an event state accumulation mechanism to update candidate navigation strategies, thereby reducing redundant processing in the continuous calculation process and improving the real-time response capability of autonomous decision-making in complex navigation environments.

[0010] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: An event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels includes the following steps: Step S1: Multi-source navigation information acquisition and synchronous processing.

[0011] The system acquires environmental perception information, hull status information, and historical navigation decision information during the autonomous navigation of the unmanned surface vessel (USV). Environmental perception information includes water surface images acquired by visual sensors; hull status information includes attitude, velocity, position, and operational status information acquired by the inertial measurement unit, positioning equipment, velocity detection unit, and onboard control system.

[0012] Multi-source information is processed synchronously based on the sampling timestamps of different sensors, and corresponding visual information sequences and ship status information sequences are constructed according to preset time windows. Preset navigation strategy categories are used as decision labels.

[0013] Step S2, Visual event feature encoding The synchronized visual information sequence is input into the visual event encoding path to perform feature extraction and event-based transformation on continuous image data.

[0014] First, the spatial feature extraction module is used to obtain shoreline areas, obstacle outlines and local texture information in the water environment. Then, the pulse coding mechanism is used to convert continuous visual features into event sequences with temporal sparsity. The relationship between different spatial regions is established through event association calculation to obtain visual event features that characterize the environmental state.

[0015] Step S3, Encoding the characteristics of ship hull status events The synchronized hull state information sequence is input into the state event encoding path to express the continuous motion state as a time-series event.

[0016] First, the multidimensional hull state parameters are mapped to a unified feature space. Then, an event neural unit with a dynamic threshold adjustment mechanism is used to accumulate and encode the current state input and historical state information. The event response threshold is adjusted according to the changes in hull motion to obtain state event features that include attitude changes, speed changes, and motion trend information.

[0017] To ensure the unified fusion of visual event features and ship state event features, this invention employs a unified event timeline for synchronous representation. Environmental change events generated during the visual event encoding process and motion change events generated during the ship state encoding process are both updated according to corresponding time windows and mapped to the same-dimensional event feature space, giving event information from different sources a unified data representation format.

[0018] Step S4, Adaptive fusion of multimodal event features Visual event features and ship state event features are mapped to a unified feature space, and adaptive fusion parameters are generated based on the two types of event features at the current time step.

[0019] The contribution ratios of visual information and ship state information in the current decision-making process are dynamically adjusted using the fusion parameters. The two types of event features are weighted and fused, and the effective information in the fusion process is maintained through feature residual mapping, forming a multimodal fusion feature oriented towards navigation strategy selection.

[0020] Unlike computational methods based on fixed weights or static attention, the fusion parameters in this invention are jointly determined by the changes in visual events and the changes in the ship's state at the current time step. When the external environment changes significantly, the fusion ratio of visual event features is increased; when the ship's motion state changes significantly, the fusion ratio of state event features is increased, thereby achieving dynamic information adjustment based on the intensity of event changes.

[0021] Step S5: Navigation strategy decision based on accumulated event states The multimodal fusion features are input into the strategy decision module, and multiple candidate strategy response units correspond to different navigation strategy categories.

[0022] Each strategy response unit performs cumulative updates based on the current fusion feature input and historical event status, and determines the strategy selection result at the current moment based on the cumulative response value.

[0023] After the preset time window ends, the response results of each candidate strategy are comprehensively evaluated, the strategy with the largest response value is selected as the current navigation decision result of the unmanned vessel, and the decision result is sent to the unmanned vessel execution system.

[0024] Beneficial effects.

[0025] Compared with the prior art, the present invention has the following beneficial effects: (1) In view of the problem that there are differences between the visual information and the hull status information of unmanned ships in terms of data form and dynamic change law, the present invention adopts the visual event coding path and the hull status event coding path to process information separately, so that the different modal information maintains independent feature expression, reduces information interference generated during direct fusion, and improves the utilization efficiency of multi-source perception information.

[0026] (2) This invention constructs a multimodal event feature adaptive fusion mechanism, which dynamically adjusts the influence ratio of different information sources according to the current navigation environment and the ship's motion state, avoiding the problem that the traditional fixed weight fusion method is not adaptable to complex and changing scenarios, and improving the reliability of autonomous navigation decision-making of unmanned ships.

[0027] (3) The present invention generates an event response only for information that meets the change conditions through an event triggering mechanism, and retains historical change information by using an event state accumulation mechanism, so that the navigation strategy can be adjusted in a timely manner according to key environmental changes and motion state changes.

[0028] (4) This invention reduces the need for redundant information processing in continuous computing by using event-based information expression and state accumulation decision-making, which is beneficial for realizing real-time autonomous decision-making on the edge computing platform of unmanned ships. Attached Figure Description

[0029] Figure 1 This is an overall flowchart of the event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to the present invention; Figure 2 This is a schematic diagram of the multi-source navigation information acquisition, synchronous processing, and dual-path event coding process of the present invention; Figure 3 This is a schematic diagram of the adaptive fusion of visual event features and ship state event features, and the navigation strategy decision-making process of the present invention. Detailed Implementation

[0030] The following detailed description, in conjunction with the accompanying drawings, illustrates an event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels proposed in this invention.

[0031] It should be noted that this embodiment is used to illustrate the technical solution of the present invention, and not to limit the scope of protection of the present invention. Those skilled in the art can make equivalent substitutions or adjustments to some technical features without departing from the technical concept of the present invention.

[0032] like Figure 1 As shown, the event-driven multimodal adaptive fusion decision-making method of the present invention includes five steps: multi-source navigation information acquisition and synchronous processing, visual event feature encoding, hull state event feature encoding, multimodal event feature adaptive fusion, and navigation strategy decision-making based on event state accumulation.

[0033] Step S1: Multi-source navigation information acquisition and synchronization processing During autonomous navigation, the unmanned surface vessel (USV) acquires navigation environment information and hull operational status information through various onboard sensors. Environmental perception information is collected by visual sensors, primarily including images of the water surface surrounding the USV. Hull status information is acquired by the inertial measurement unit, positioning equipment, velocity detection unit, and onboard control system, including the USV's attitude, velocity, position, and operational status. Let the time window length be... Visual sequence is denoted as The physical state sequence is denoted as ,in .

[0034] Since different sensors have different data sampling frequencies and time response characteristics, the multi-source information acquired is processed synchronously according to the sampling timestamps of each sensor, so that visual information and ship status information maintain a correspondence in the time dimension.

[0035] Furthermore, the synchronized multi-source information is divided according to a preset time window to construct visual information sequences and ship state information sequences respectively. Corresponding navigation strategy category labels are established based on the navigation strategy selection results during the actual operation of the unmanned ship, providing input data for subsequent event feature encoding and decision model processing.

[0036] Step S2, Visual event feature encoding like Figure 2 As shown, the visual information sequence obtained in step S1 is input into the visual event encoding pathway to perform event-based feature representation on continuous image data.

[0037] First, spatial features are encoded for continuous visual information, and effective response information of changing regions is retained by combining event triggering mechanisms to avoid repeated calculations of continuous stable regions. Then, based on the process of membrane potential change of spiking neurons, visual features with changing responses are converted into temporally sparse event sequences.

[0038] Specifically, based on the changes in the membrane potential of spiking neurons, the input visual features are cumulatively updated over time. When the membrane potential reaches a preset threshold, an event response is generated, thus completing the conversion of continuous visual information into event information. For the image at time step t... Two-dimensional convolution and layer normalization are performed to obtain local visual features. : (1) Will Input is given to a spiking neuron, which updates its membrane potential and generates pulses according to the following formula: (2) (3)

[0039] in, This represents the membrane potential of a visual spiking neuron at time step t. The membrane potential attenuation coefficient is... The threshold for issuance, H(·) is the output pulse, and H(·) is the pulse firing function.

[0040] Furthermore, event correlation calculations are performed on the obtained visual event sequences. By establishing correlations between different spatial regions, the ability to express spatial environment information is enhanced, and visual event features characterizing the external environment state of the unmanned vessel are obtained. The impulse features are divided into N spatial labels and mapped to... .Depend on Query features were obtained respectively Key features Sum value characteristics Calculate the impulse attention response And obtain visual spatial features : (4) (5) Through the above steps, visual information is transformed from continuous and dense data into event features with temporal sparsity, providing an environmental perception basis for subsequent multimodal fusion decision-making.

[0041] Step S3, Encoding the characteristics of ship hull status events like Figure 2 As shown, the hull state information sequence obtained in step S1 is input into the state event encoding path to express the continuous motion state of the unmanned vessel as a time-series event.

[0042] First, the multi-dimensional hull state parameters are mapped to a unified feature space, giving different types of state information a consistent feature representation. For the physical state vector at time step t... Perform feature mapping: (6) Subsequently, the mapped state features are input into an event neural unit with a dynamic threshold adjustment mechanism to accumulate and encode the current state input and historical state information.

[0043] Specifically, the event response threshold is dynamically adjusted based on changes in the hull state, enabling the event encoding process to generate corresponding responses according to changes in the unmanned vessel's motion state. The mapping characteristics, together with the pulse state of the previous time step, form the input current of the cyclic pulse unit. (7) The membrane potential, adaptation variable, and dynamic firing threshold of the cyclic pulse unit are updated using the following formula: (8) (9) (10)

[0044] in, To adapt variables, To adapt to the attenuation coefficient of the variable, Based on the threshold, This is the threshold adjustment coefficient. When neurons fire continuously, the dynamic threshold increases; when neurons are silent, the dynamic threshold gradually recovers. This mechanism can simultaneously maintain sensitivity to short-term changes and long-term motor state memory.

[0045] Through the above process, state event features containing information on unmanned vessel attitude changes, speed changes, and motion trends are obtained, enabling the event-based expression of the vessel's dynamic state.

[0046] Step S4, Adaptive fusion of multimodal event features like Figure 3 As shown, the visual event features obtained in step S2 and the ship state event features obtained in step S3 are mapped to a unified feature space respectively.

[0047] Since visual information mainly reflects changes in the external environment, while hull state information mainly reflects its own motion state, the two types of information have different degrees of importance in the autonomous navigation decision-making process of unmanned ships. Therefore, this invention generates adaptive fusion parameters based on the visual event features and state event features of the current time step.

[0048] Specifically, the change response intensity of visual event features and ship state event features at the current time step relative to the historical time state is first calculated, and adaptive fusion parameters are generated based on the normalized results of the change intensity of the two types of events. The fusion parameters can reflect the degree of change of the external environment and the degree of change of the ship's motion state during the current navigation process. When the change intensity of visual events is higher than that of state events, the fusion weight corresponding to visual event features is increased; when the change intensity of ship state is high, the fusion weight corresponding to ship state event features is increased.

[0049] Visual spatial features at the same time step and state-time features Perform projection and normalization separately: (11) The gating coefficient is generated by concatenating the two feature paths. And form a weighted fusion feature. : (12) (13) Furthermore, residual mapping is performed on the fused multimodal features to maintain the integrity of effective feature information during the fusion process: (14)

[0050] The above method enables adaptive fusion of multimodal event features in response to changes in the current navigation state, forming multimodal fusion features oriented towards navigation strategy selection.

[0051] Step S5: Navigation strategy decision based on accumulated event states like Figure 3 As shown, the multimodal fusion features obtained in step S4 are input into the strategy decision module.

[0052] The strategy decision-making module includes multiple event response units, each corresponding to a preset navigation strategy category. Each event response unit updates its internal state variables based on the input multimodal fusion event features and generates a corresponding strategy response value based on the accumulated state variable results.

[0053] Each policy response unit accumulates and updates based on the multimodal fusion features input at the current time step and the historical event states, forming the corresponding policy response result through the event state change process.

[0054] Specifically, the accumulated response value is updated based on the internal state change process of the strategy response unit. The current is mapped to the input current of the action readout layer, and each action readout neuron updates its membrane potential over time. (15) (16) (17)

[0055] After the preset time window ends, the cumulative response results of each candidate strategy response unit are comprehensively evaluated, and the strategy with the largest response value is selected as the current unmanned surface vessel navigation decision. Specifically, the membrane potentials of K action readout neurons are averaged over time to obtain the action probability and predicted action: (18) The average membrane potential preserves accumulated state information even before the firing threshold is reached, avoiding information loss caused by simply counting the number of pulses. When the environment or motion state changes, the input current of the readout neuron corresponding to the new action increases, and its membrane potential can quickly exceed the response corresponding to the original action, thus completing the action switch. Finally, the determined navigation decision is sent to the unmanned surface vessel execution system, and the actuator performs the corresponding navigation operation based on the decision. Experimental verification

[0056] To verify the effectiveness of the event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels proposed in this invention, experimental tests were conducted using actual collected multimodal data of unmanned vessel navigation on a lake.

[0057] The experiment mainly verifies the technical effects of the method of the present invention in terms of multi-source information fusion, accuracy of navigation strategy decision-making, and ability to express event-driven features.

[0058] (1) Experimental data and experimental conditions The experiment used a self-collected multimodal navigation dataset of unmanned surface vessels (USVs) on a lake. This dataset contains visual image information, hull status information, and corresponding navigation strategy labels collected synchronously during the autonomous navigation of the USV.

[0059] The visual information is collected by the camera equipment carried by the unmanned vessel and is used to describe the external environmental features such as the water surface environment, navigation boundaries and obstacles; the hull status information includes multiple physical variables that can reflect the motion state of the unmanned vessel, such as attitude parameters, velocity parameters, position parameters and operating status parameters.

[0060] The original dataset contains 2664 consecutive navigation records, each corresponding to a frame of water surface image and synchronously acquired hull status information. Multiple navigation strategy categories are set as model output labels based on the actual operational requirements of the unmanned surface vessel.

[0061] During the experiment, data from 10 consecutive time steps were used as input windows to predict the navigation strategy category for the next time step, resulting in 2,654 valid time window samples.

[0062] To avoid the impact of data correlation between adjacent time windows on the experimental results, the data was divided into non-overlapping sets according to the original collection order. The training set contained 1258 sample windows, the validation set contained 388 sample windows, and the test set contained 408 sample windows. There was no overlap between the input data or the target data for prediction among the data sets.

[0063] (2) Comparative experiment To verify the effectiveness of the method of this invention in the navigation strategy decision-making task of unmanned vessels, several typical multimodal spatiotemporal models were selected as comparison methods, including: ① ResNet18+Transformer method; ② MobileNetV3-Small+GRU method; ③ ViT Transformer method; ④ CNN+LSTM method.

[0064] The above comparison methods all use the same data input format, data partitioning method, and evaluation metrics as this invention to ensure the fairness of the experimental comparison. Evaluation metrics include decision accuracy, balance accuracy, and Macro-F1 score. Experimental results are shown in Table 1.

[0065] Table 1 Comparison of navigation strategy decision-making performance of different methods As shown in Table 1, the method of the present invention achieves a decision accuracy of 83.82%, a balance accuracy of 83.76%, and a Macro-F1 accuracy of 84.72% on the test set, all of which are superior to other comparative methods.

[0066] Experimental results show that the method of the present invention can effectively utilize visual environment information and hull state information, improve the accuracy of unmanned vessel navigation strategy prediction through event-driven coding, adaptive fusion and state accumulation decision-making mechanism, and enhance the stability of autonomous decision-making in complex water environment.

[0067] (3) Module validity verification experiment To verify the contribution of the key technical modules proposed in this invention to decision performance, further ablation experiments were conducted. The experiments removed: ① the event-driven pulse coding mechanism and ② the visual event feature attention modeling mechanism, and compared them with the complete model. Specifically: after removing the event-driven pulse mechanism, a continuous value neural network structure was used instead; after removing the visual event association modeling mechanism, only a basic pulse convolution structure was used to extract event features.

[0068] The experimental results are shown in Table 2.

[0069] Table 2 Impact of different module combinations on decision performance Method of the present invention √ √ 83.82 83.76 84.72 Remove event-driven impulse mechanism × √ 66.42 69.03 72.06 Remove visual event association modeling √ × 67.16 68.79 69.33 As shown in Table 2, when the event-driven impulse mechanism was removed, the model accuracy decreased from 83.82% to 66.42%, a decrease of 17.40 percentage points; when the visual event association modeling mechanism was removed, the model accuracy decreased to 67.16%, a decrease of 16.67 percentage points.

[0070] Experimental results show that the event-driven pulse coding mechanism in this invention can effectively utilize the temporal state change information during the navigation process of unmanned vessels and improve the dynamic state expression capability; the visual event feature modeling mechanism can enhance the spatial information extraction capability in complex water surface environments and improve the contribution of visual information to navigation strategy decision-making.

[0071] (4) Analysis of experimental verification results Based on the above experimental results, it can be seen that the event-driven multimodal adaptive fusion decision-making method proposed in this invention can effectively fuse visual environment information and hull state information of unmanned vessels.

[0072] By using a dual-path event encoding method, different modal information can maintain independent feature expression before fusion, reducing information interference caused by direct fusion of heterogeneous data; By using a multimodal event feature adaptive fusion mechanism, the proportion of different information sources is dynamically adjusted according to the current navigation status, thereby improving the reliability of decision-making in complex environments. By using an event state accumulation decision-making mechanism, the navigation strategy selection is completed by accumulating states over time, which improves the strategic judgment ability of unmanned ships during autonomous navigation.

[0073] Therefore, the method of the present invention can improve the ability to utilize multi-source information and the accuracy of navigation strategy decision-making during the autonomous navigation of unmanned vessels, and has good engineering application value.

Claims

1. An event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels, characterized in that, Includes the following steps: S1. Multi-source navigation information acquisition and synchronous processing: Acquire environmental perception information, hull status information, and historical navigation decision information during the autonomous navigation of the unmanned vessel; The environmental perception information includes water surface image information collected by a visual sensor; the hull status information includes attitude information, speed information, position status information, and operating status information obtained by an inertial measurement unit, positioning device, speed detection unit, and shipboard control system. The environmental perception information and the ship's status information are processed synchronously based on the sampling timestamps of different sensors, and visual information sequences and ship status information sequences are constructed respectively according to preset time windows. S2, Visual Event Feature Encoding: The synchronized visual information sequence is input into the visual event encoding pathway to express event-based features in continuous image data. Spatial feature encoding is used to extract shoreline area features, obstacle outline features, and local texture features in the water surface environment, and continuous visual features are converted into a sequence of visual events with temporal sparsity based on an event triggering mechanism. Further, event correlation calculations are performed on the visual event sequence to establish correlation relationships between different spatial regions and obtain visual event features that characterize the external environment of the unmanned vessel. S3, Ship Status Event Feature Encoding: The synchronized hull state information sequence is input into the state event encoding path to express the continuous motion state of the unmanned vessel as a time-series event. Multidimensional hull state parameters are mapped to a unified feature space and input into an event neural unit with a dynamic threshold adjustment mechanism. The event response threshold is adjusted according to the changes in hull motion state. The current state input and historical state information are cumulatively encoded to obtain state event features containing attitude changes, speed changes and motion trend information. S4. Adaptive fusion of multimodal event features: The visual event features and the state event features are respectively mapped to a unified feature space and synchronously represented using a unified event timeline; Adaptive fusion parameters are generated based on the changes in visual events and the changes in hull state at the current time step. The contribution ratio of visual event features and state event features in the current decision-making process is dynamically adjusted according to the adaptive fusion parameters. The two types of event features are weighted and fused, and the effective feature information in the fusion process is maintained through residual mapping to obtain multimodal fusion features. S5. Navigation strategy decision based on event state accumulation: The multimodal fusion features are input into the strategy decision module; Multiple event response units correspond to different preset navigation strategy categories. Each event response unit updates its internal state variables based on the current input multimodal fusion features and historical event states, and generates the corresponding strategy response value based on the accumulated state variable results. After the preset time window ends, the strategy response results generated by each event response unit are evaluated, the strategy with the largest response value is selected as the current unmanned vessel navigation decision result, and sent to the unmanned vessel execution system.

2. The event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to claim 1, characterized in that: The event triggering mechanism in the visual event encoding pathway is based on the process of membrane potential change in spiking neurons. The input visual features are cumulatively updated over time, and an event response is generated when the membrane potential reaches a preset emission threshold, thereby realizing the conversion of continuous visual information into visual event information.

3. The event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to claim 1, characterized in that: The event association calculation of the visual event features includes: The visual event sequence is divided into multiple spatial labels, and query features, key features, and value features are obtained by mapping them respectively; The event association response is calculated based on the query features, key features, and value features to enhance the information association capability between different spatial regions.

4. The event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to claim 1, characterized in that: The dynamic threshold adjustment mechanism in the state event encoding path includes: Adjust the event response threshold of the event neural unit according to changes in the ship's condition; When the event neural unit generates continuous responses, the event response threshold is increased; when the event neural unit is in a silent state, the event response threshold is gradually restored.

5. The event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to claim 1, characterized in that: The unified event timeline is used to synchronously update environmental change events generated during the visual event encoding process and motion change events generated during the hull state encoding process. The visual event features and state event features are updated according to the corresponding time windows and mapped to the event feature space of the same dimension to achieve a unified expression of event information from different sources.

6. The event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to claim 1, characterized in that: The process of generating the adaptive fusion parameters includes: Calculate the change response intensity of the visual event features at the current time step relative to the historical state, and the change response intensity of the hull state event features relative to the historical state, respectively. The adaptive fusion parameters are generated based on the normalized results of the response intensities of the two types of changes.

7. The event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to claim 6, characterized in that: When the intensity of visual event changes is higher than the intensity of changes in the ship's state, increase the fusion ratio corresponding to the visual event features. When the intensity of changes in the ship's state is higher than the intensity of changes in visual events, the fusion ratio corresponding to the state event features is increased.

8. The event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to claim 1, characterized in that: The event response unit in the strategy decision-making module updates its internal state variables based on multimodal fusion features and uses the event state accumulation process to form a strategy response result. The strategy response result is determined based on the cumulative state of the event response unit.

9. The event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to claim 8, characterized in that: When determining the navigation decision outcome, the cumulative response results of the event response units corresponding to multiple candidate strategies are comprehensively evaluated. The corresponding navigation strategy category is determined by calculating the status response results of each strategy response unit within a preset time window.

10. An event-driven multimodal adaptive fusion decision-making method for autonomous navigation of unmanned vessels according to any one of claims 1 to 9, characterized in that: The navigation decision results include the navigation control strategy determined by the unmanned vessel based on the current external environment information and its own motion state, and the corresponding navigation operation is completed by the unmanned vessel execution system according to the navigation control strategy.

Images