A large model-based situation awareness method and system in an air, land and sea integrated maritime combat scene

Abstract

The application provides a water, land and air integrated maritime combat scene situation awareness method and system based on a large model, relates to the field of military combat, and comprises the following steps: acquiring water, land and air multi-source combat data and performing space-time alignment processing; performing parallel feature extraction on the data aligned by each mode, and performing dimension reduction on the extracted features through an adaptive bottleneck layer of the large model; based on the feature vectors of each mode after dimension reduction, constructing a multi-level attention bottleneck unit in the large model, and respectively distributing each mode feature vector to a corresponding level for attention weight calculation; realizing dynamic fusion of different mode feature vectors through feature splicing and an attention mechanism, inputting multi-modal fusion information into a classifier of the large model for battlefield situation assessment, and generating a dynamic strike decision scheme according to the assessment result. The application is used to solve the defects in the prior art that multi-source heterogeneous data processing exists inter-modal conflicts, and it is difficult to realize efficient situation awareness and accurate strike decision.

Description

A situational awareness method and system for integrated maritime, land, and air combat scenarios based on a large model. Technical Field

[0001] This invention relates to the field of military operations, and in particular to a situational awareness method and system for integrated land, sea and air combat scenarios based on a large model. Background Technology

[0002] In modern naval warfare, multi-service joint operations involving land, sea, and air have become a trend. However, due to the modal differences in multi-source heterogeneous data from surface ships, aircraft, and land command systems (such as radar detection data, infrared image data, and voice communication data), conflicts can easily arise between different modalities, making it difficult to achieve situational awareness and strike decisions quickly and accurately. Traditional combat decision-making methods cannot efficiently process massive amounts of complex data and cannot meet the real-time and precision requirements of modern warfare.

[0003] However, existing technologies suffer from intermodal conflicts when integrating heterogeneous data from multiple sources such as radar, satellites, drones, and crew terminals. For example, the timestamps of meteorological data and ship sensor data are not synchronized, making it difficult to achieve efficient situational awareness and precision strikes. Summary of the Invention

[0004] This invention provides a situational awareness method and system for integrated land, sea and air combat scenarios based on a large model, in order to solve the shortcomings of existing technologies in processing multi-source heterogeneous data, which suffers from inter-modal conflicts and makes it difficult to achieve efficient situational awareness and precise strike decision-making.

[0005] On the one hand, this invention provides a situational awareness method for integrated land, sea, and air maritime combat scenarios based on a large model, including:

[0006] The system acquires multi-source combat data from land, sea, and air and performs spatiotemporal alignment processing to obtain data aligned to each mode. The multi-source combat data from land, sea, and air includes radar detection data from surface ships, infrared image data from airborne aircraft, and communication command data from land command systems.

[0007] The data after each modality alignment is input into the multimodal encoder of the large model for parallel feature extraction, and the extracted features are reduced in dimensionality through the adaptive bottleneck layer of the large model to obtain the feature vectors of each modality after dimensionality reduction.

[0008] Based on the feature vectors after dimensionality reduction of each modality, a multi-level attention bottleneck unit is constructed in the large model. The feature vector of each modality is assigned to the corresponding level for attention weight calculation, and updated bottleneck unit parameters are generated.

[0009] Cross-modal information interaction is performed in the updated bottleneck unit, and dynamic fusion of feature vectors of different modalities is achieved through feature concatenation and attention mechanism to generate multimodal fusion information containing spatiotemporal correlation.

[0010] The multimodal fusion information is input into the classifier of the large model for battlefield situation assessment, and a dynamic strike decision plan is generated based on the assessment results.

[0011] Furthermore, multi-source combat data from land, sea, and air is acquired and spatiotemporally aligned to obtain modally aligned data. This multi-source combat data includes radar detection data from surface ships, infrared image data from airborne aircraft, and communication command data from land-based command systems, including:

[0012] The preprocessing module of the large model standardizes and normalizes multi-source combat data from different sources (land, sea, and air) to generate preprocessed data in a unified data format.

[0013] The preprocessed data is subjected to time alignment, and the timestamps of different data sources are matched based on the time series dynamic time warping algorithm to generate time-synchronized intermediate data.

[0014] The intermediate data is spatially aligned by converting the coordinate system of the multi-source combat data (land, sea, and air) to a preset global spatial reference system, thereby generating spatially aligned intermediate data.

[0015] Intermodal correlation matching is performed between the time-synchronized intermediate data and the spatially aligned intermediate data to generate modally aligned data with spatiotemporal consistency constraints.

[0016] Furthermore, intermodal correlation matching is performed between the time-synchronized intermediate data and the spatially aligned intermediate data to generate modally aligned data for each modality, including spatiotemporal consistency constraints, such as:

[0017] Radar pulse time series, satellite scan time series and UAV video frame time series are extracted from preprocessed data to construct a multi-source heterogeneous time series set.

[0018] The cumulative distance matrix between any two time series in the multi-source heterogeneous time series set is calculated using a dynamic time warping algorithm, and the optimal alignment path is determined through a backtracking mechanism.

[0019] The time series is subjected to nonlinear interpolation based on the optimal alignment path to eliminate timing deviations caused by sampling frequency differences and generate a time-aligned synchronization sequence.

[0020] The spatial alignment operation includes:

[0021] Establish a global spatial coordinate system centered on ships in maritime combat scenarios, and calculate the transformation matrix from radar polar coordinates, satellite geographic coordinates, and UAV visual coordinates to the global coordinate system;

[0022] Spatial location information from different data sources is mapped to the same spatial reference frame using a coordinate transformation matrix, and spatial measurement noise is eliminated using a Kalman filter algorithm to generate spatially aligned high-confidence location data.

[0023] Furthermore, the aligned data of each modality is input into the multimodal encoder of the large model for parallel feature extraction, and the extracted features are dimensionality reduced through the adaptive bottleneck layer of the large model to obtain the dimensionality-reduced feature vectors of each modality, including:

[0024] A radar point cloud feature encoder was constructed, and a three-dimensional sparse convolutional network was used to extract the spatial distribution features of obstacles around the ship.

[0025] A satellite remote sensing feature encoder was constructed, and a multispectral attention network was used to extract the spatiotemporal evolution features of the sea surface temperature field and ocean current vector field;

[0026] A UAV video feature encoder was constructed, and a spatiotemporal transformer network was used to extract continuous inter-frame motion features of abnormal ship attitude behavior;

[0027] A text semantic feature encoder was constructed, and a pre-trained language model was used to extract keyword entities and sentiment features from crew report texts.

[0028] The adaptive bottleneck layer performs dimensionality reduction on the extracted features, including:

[0029] By using a learnable projection matrix, each modal feature is mapped to a low-dimensional subspace, and a gating mechanism is used to dynamically adjust the compression ratio of each modal feature, thereby generating a dimensionality-reduced feature vector that retains key information.

[0030] Furthermore, based on the dimensionality-reduced feature vectors of each modality, a multi-level attention bottleneck unit is constructed in the large model. Each modality feature vector is assigned to its corresponding level for attention weight calculation, generating updated bottleneck unit parameters, including:

[0031] A hierarchical attention mechanism is deployed in the bottleneck unit. The bottom attention layer calculates the correlation weight of features within the modality, the middle attention layer calculates the complementarity weight of features across modalities, and the top attention layer calculates the contribution weight of global features.

[0032] The updated bottleneck unit parameters include:

[0033] A soft routing algorithm is used to calculate the allocation probability of each modality feature vector to different attention layers, and the feature vectors are weighted and recombined according to the probability distribution.

[0034] In each attention layer, the recombined feature vector is concatenated with the learnable positional encoding, and the state parameters of the bottleneck unit are updated through residual connections.

[0035] Furthermore, cross-modal information interaction is performed in the updated bottleneck unit, and dynamic fusion of feature vectors from different modalities is achieved through feature concatenation and attention mechanisms to generate multimodal fusion information containing spatiotemporal correlations, including:

[0036] A dynamic graph structure is constructed with modal features as nodes and cross-modal associations as edges. A graph attention network is used to iteratively update the message passing paths between nodes.

[0037] In each graph update iteration, nonlinear spatial dependencies between cross-modal features are captured by deformable convolutional kernels, and feature interaction patterns of different subspaces are fused using a multi-head attention mechanism.

[0038] The node features that have undergone multiple rounds of graph updates are aggregated using a dual-path aggregation method of max pooling and average pooling to generate multimodal fusion information that retains both local details and global context.

[0039] Furthermore, the multimodal fusion information is input into the classifier of the large model for battlefield situation assessment, and a dynamic strike decision scheme is generated based on the assessment results, including:

[0040] Based on the battlefield situation assessment results, a multi-objective optimization algorithm is used to solve the optimal fire allocation scheme under the constraints of combat resources.

[0041] A flight path planning model is constructed, and a four-dimensional threat field is built by integrating battlefield environmental data. A flight path is generated by an ant colony algorithm. The four-dimensional threat field includes air threats, surface threats, underwater threats, and electromagnetic threats.

[0042] Based on the ship type and the status of combat resource inventory, knowledge graph reasoning technology is used to match the optimal combination of combat equipment and generate a phased mission execution instruction set.

[0043] On the other hand, a situational awareness system for integrated land, sea, and air maritime combat scenarios based on a large model includes:

[0044] The acquisition module is used to acquire multi-source combat data from land, sea, and air and perform spatiotemporal alignment processing to obtain data aligned to each mode. The multi-source combat data from land, sea, and air includes radar detection data from surface ships, infrared image data from airborne aircraft, and communication command data from land command systems.

[0045] The processing module is used to input the aligned data of each modality into the multimodal encoder of the large model for parallel feature extraction, and to reduce the dimensionality of the extracted features through the adaptive bottleneck layer of the large model to obtain the dimensionality-reduced feature vectors of each modality; based on the dimensionality-reduced feature vectors of each modality, a multi-level attention bottleneck unit is constructed in the large model, and each modal feature vector is assigned to the corresponding level for attention weight calculation to generate updated bottleneck unit parameters; cross-modal information interaction is performed in the updated bottleneck unit, and the dynamic fusion of feature vectors of different modalities is realized through feature concatenation and attention mechanism to generate multimodal fusion information containing spatiotemporal correlation; the multimodal fusion information is input into the classifier of the large model for battlefield situation assessment, and a dynamic strike decision scheme is generated based on the assessment results.

[0046] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the situational awareness method for integrated land, sea and air combat scenarios based on a large model as described above.

[0047] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the situational awareness method for integrated land, sea and air combat scenarios based on a large model as described above.

[0048] On the other hand, the present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the situational awareness method for integrated land, sea and air combat scenarios based on a large model as described above.

[0049] This invention provides a situational awareness method and system for integrated land, sea, and air maritime combat scenarios based on a large model. By processing multi-source heterogeneous data such as radar detection data, infrared images, and communication commands through spatiotemporal alignment, it solves the spatiotemporal deviation problem caused by sensor differences in traditional combat, improving battlefield situational awareness accuracy and achieving centimeter-level positioning and millisecond-level time synchronization. It utilizes multi-level attention bottleneck units to achieve cross-modal semantic alignment, reducing operational misjudgment rate and feature redundancy rate. Furthermore, through dynamic weight allocation and cross-modal semantic consistency preserved by adaptive bottleneck layers, it significantly improves the accuracy of analysis under complex sea conditions. At the resource optimization level, it improves operational resource utilization and reduces operational failure rate, while also reducing operational costs through multi-objective optimization. At the scenario adaptability level, the system supports full-modal redundant perception under extreme sea conditions such as typhoons, strong currents, and low visibility, improving full sea state coverage. This achieves efficient battlefield situational awareness and accurate strike decision generation, enhancing the operational effectiveness of integrated land, sea, and air maritime operations and the scientific and timely nature of command and decision-making. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0051] Figure 1 is a flowchart illustrating the situational awareness method for integrated land, sea, and air combat scenarios based on a large model provided in an embodiment of the present invention.

[0052] Figure 2 is a schematic diagram of a situational awareness system based on a large model for integrated land, sea and air maritime combat scenarios provided in an embodiment of the present invention.

[0053] Figure 3 is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0055] Figure 1 is one of the flowcharts of the situational awareness method for integrated land, sea and air combat scenarios based on a large model provided in an embodiment of the present invention.

[0056] As shown in Figure 1, the situational awareness method for integrated land, sea, and air combat scenarios based on a large model provided in this embodiment of the invention mainly includes the following steps:

[0057] 11. Acquire multi-source combat data from land, sea, and air and perform spatiotemporal alignment processing to obtain data aligned to each mode. The multi-source combat data from land, sea, and air includes radar detection data from surface ships, infrared image data from airborne aircraft, and communication command data from land command systems.

[0058] 12. Input the aligned data of each modality into the multimodal encoder of the large model for parallel feature extraction, and reduce the dimensionality of the extracted features through the adaptive bottleneck layer of the large model to obtain the dimensionality-reduced feature vectors of each modality.

[0059] 13. Based on the feature vectors after dimensionality reduction of each modality, construct multi-level attention bottleneck units in the large model, and assign each modality feature vector to the corresponding level for attention weight calculation to generate updated bottleneck unit parameters.

[0060] 14. Perform cross-modal information interaction in the updated bottleneck unit, and realize the dynamic fusion of feature vectors of different modalities through feature splicing and attention mechanism to generate multimodal fusion information containing spatiotemporal correlation;

[0061] 15. Input the multimodal fusion information into the classifier of the large model to perform battlefield situation assessment, and generate a dynamic strike decision plan based on the assessment results.

[0062] In this embodiment of the invention, spatiotemporal alignment is performed on different modal data such as surface ship radar, airborne aircraft infrared images, and land-based command and communication commands to eliminate spatiotemporal deviations caused by sensor differences, ensure data consistency, solve the data silo problem in traditional combat systems, and achieve unified representation of battlefield information across the entire domain. A multimodal encoder is used to extract features from different data modalities in parallel, improving computational efficiency. An adaptive bottleneck layer intelligently reduces the dimensionality of high-dimensional features, reducing redundant information and improving subsequent computation speed. Through a submodal attention mechanism, data from different sensors are dynamically weighted to enhance the identification capability of key targets. The cross-modal attention mechanism mines the implicit correlations between different data (such as the coordinated changes in ship trajectories and aircraft infrared signals), improving the depth of battlefield situation understanding. Through feature stitching and a multi-head attention mechanism, deep fusion of water, land, and air data is achieved to generate battlefield situation information with spatiotemporal correlation. Even when electronic warfare interference or partial sensor failure occurs, high-precision perception can still be maintained through cross-modal compensation. Based on the fused information, a large-model classifier quickly assesses the battlefield situation, generates the optimal strike plan, dynamically adjusts combat strategies, and adapts to the rapidly changing battlefield environment.

[0063] As shown in Figure 1, 11, multi-source combat data from land, sea, and air is acquired and spatiotemporally aligned to obtain data aligned to each mode. This multi-source combat data includes radar detection data from surface ships, infrared image data from airborne aircraft, and communication command data from land-based command systems, including:

[0064] 111. The preprocessing module of the large model is used to standardize and normalize multi-source combat data from different sources (land, sea, and air) to generate preprocessed data in a unified data format.

[0065] 112. Perform time alignment operation on the preprocessed data, and match the timestamps of different data sources based on the time series dynamic time warping algorithm to generate time-synchronized intermediate data;

[0066] 113. Perform spatial alignment on the intermediate data to transform the coordinate system of the multi-source combat data (land, sea, and air) to a preset global spatial reference system, and generate spatially aligned intermediate data.

[0067] 114. Perform intermodal correlation matching between the time-synchronized intermediate data and the spatially aligned intermediate data to generate modally aligned data with spatiotemporal consistency constraints.

[0068] In this embodiment of the invention, step 111 eliminates the dimensional differences between different sensor data (such as radar distance / velocity, infrared image pixel intensity, and communication command text format) to make them comparable. For example, the radar detection range is normalized to [0,1], the infrared image is histogram equalized, and the communication command is converted into a structured vector. Step 112 solves the timing misalignment problem caused by different sensor sampling frequencies (such as radar 10Hz, infrared image 30fps, and communication command 1Hz). A dynamic time warping algorithm is used to align asynchronous timestamps to ensure that the time points of "radar detects target" and "infrared image captures target" are consistent. Step 113 unifies the coordinate systems of different sensors. (e.g., the local coordinate system of a ship, the inertial navigation system of a fighter jet, and the geographic coordinate system of a command system) Through coordinate transformation, all data are mapped to a global reference system to achieve spatial position matching between "ship radar targets" and "fighter jet infrared targets". Step 114, based on spatiotemporal consistency constraints, associates the same target in different modalities, associating "a suspicious target detected by a ship radar" and "a heat source in a fighter jet infrared image" as the same enemy unit. Spatiotemporal alignment processing is a core preliminary step in multimodal situational awareness. Through standardization, time synchronization, spatial registration, and cross-modal association, it solves the problem of "spatiotemporal fragmentation" of heterogeneous data, providing a highly consistent and reliable data foundation for subsequent intelligent decision-making.

[0069] As shown in Figure 1, 114. Modal correlation matching is performed between the time-synchronized intermediate data and the spatially aligned intermediate data to generate modally aligned data containing spatiotemporal consistency constraints, including:

[0070] 1141. Extract radar pulse time series, satellite scan time series and UAV video frame time series from preprocessed data to construct a multi-source heterogeneous time series set;

[0071] 1142. The cumulative distance matrix between any two time series in the multi-source heterogeneous time series set is calculated using the dynamic time warping algorithm, and the optimal alignment path is determined through a backtracking mechanism;

[0072] 1143. Perform nonlinear interpolation on the time series according to the optimal alignment path to eliminate timing deviations caused by sampling frequency differences and generate a time-aligned synchronization sequence.

[0073] The spatial alignment operation includes:

[0074] Establish a global spatial coordinate system centered on ships in maritime combat scenarios, and calculate the transformation matrix from radar polar coordinates, satellite geographic coordinates, and UAV visual coordinates to the global coordinate system;

[0075] Spatial location information from different data sources is mapped to the same spatial reference frame using a coordinate transformation matrix, and spatial measurement noise is eliminated using a Kalman filter algorithm to generate spatially aligned high-confidence location data.

[0076] In this embodiment of the invention, dynamic time warping solves the problem of inconsistent sampling frequencies of different sensors and eliminates timing deviations through nonlinear alignment; optimal path backtracking finds the best matching relationship between time series, such as whether the target detected by radar and the target in the UAV video are the same entity; nonlinear interpolation performs high-precision interpolation on low-frequency data to align it with the timestamp of high-frequency data; using the ship as the reference origin avoids the impact of Earth curvature or GPS errors on local combat; radar (polar coordinates), satellite (latitude and longitude), and UAV (visual SLAM coordinates) are uniformly transformed to the same spatial reference system; Kalman filtering denoising eliminates sensor measurement noise and improves the confidence of position data; combining the results of time alignment and spatial alignment, the same target in different modalities is associated through spatiotemporal consistency constraints, associating the "suspicious target trajectory detected by radar" and the "moving object in the UAV video" as the same enemy unit; achieving accurate spatiotemporal association of multi-source heterogeneous data, ensuring that data collected by different sensors such as radar, satellite, and UAV are strictly aligned in time and space, thereby supporting subsequent multimodal fusion and intelligent decision-making.

[0077] As shown in Figure 1, 12, the aligned data of each modality is input into the multimodal encoder of the large model for parallel feature extraction, and the extracted features are reduced in dimensionality through the adaptive bottleneck layer of the large model to obtain the dimensionality-reduced feature vectors of each modality, including:

[0078] 121. Construct a radar point cloud feature encoder and use a three-dimensional sparse convolutional network to extract the spatial distribution features of obstacles around the ship;

[0079] 122. Construct a satellite remote sensing feature encoder and use a multispectral attention network to extract the spatiotemporal evolution features of the sea surface temperature field and ocean current vector field;

[0080] 123. Construct a UAV video feature encoder and use a spatiotemporal transformer network to extract continuous inter-frame motion features of abnormal ship attitude behavior;

[0081] 124. Construct a text semantic feature encoder and use a pre-trained language model to extract keyword entities and sentiment features from crew report texts;

[0082] The adaptive bottleneck layer performs dimensionality reduction on the extracted features, including:

[0083] By using a learnable projection matrix, each modal feature is mapped to a low-dimensional subspace, and a gating mechanism is used to dynamically adjust the compression ratio of each modal feature, thereby generating a dimensionality-reduced feature vector that retains key information.

[0084] In this embodiment of the invention, a modal-specific encoder is used for deep decoupling and accurate extraction of cross-modal feature semantics. Addressing the high sparsity and strong noise interference of radar point cloud data, a three-dimensional sparse convolutional kernel is employed to model the spatial topology of obstacles surrounding the ship. By dynamically adjusting the receptive field of the convolutional kernel (adaptively changing according to the obstacle density gradient), multi-scale spatial distribution features are extracted, improving obstacle recognition accuracy and feature extraction efficiency compared to traditional voxelization methods. This provides sub-meter-level spatial semantic features for collision avoidance path planning. Furthermore, a spatiotemporal dual-channel attention module is constructed based on multispectral satellite data, utilizing the synergy of spectral and temporal attention... This study extracts the spatiotemporal coupling evolution characteristics of sea surface temperature field and ocean current vector field to reduce typhoon path prediction errors and improve the accuracy of ocean current countercurrent zone identification, providing a dynamic environmental feature foundation for combat evolution trend analysis. For the continuous inter-frame motion characteristics of UAV video streams, a spatiotemporal attention module in a spatiotemporal transformer is used to extract high-order motion features of abnormal ship attitude behavior through joint modeling of spatial and temporal attention. This improves the sensitivity of attitude anomaly detection and reduces computational cost compared to traditional optical flow methods, providing real-time dynamic feature input for capsizing risk early warning. Finally, a dual-tower feature extractor is constructed based on crew report text, and key features are extracted using a BERT model. This approach combines keyword entity and sentiment features with cross-modal contrastive learning to align text features with spatial location and temporal features, improving the accuracy of text semantic association with combat scenarios and providing multimodal mutual verification features for combat type identification. A parameter-shared projection matrix maps each modality feature to a low-dimensional subspace, and orthogonal constraints between modalities are introduced to eliminate feature redundancy, reducing the feature redundancy rate. Simultaneously, subspace similarity matching enhances modal complementarity, improving combat analysis coverage. Based on feature entropy and task relevance, a gating unit dynamically adjusts the compression ratio of each modality feature, generating a dimensionality-reduced feature vector that retains key information, thus improving the F-value of the dimensionality-reduced features. The 1-score loss is less than 3%, and the feature computation is reduced, providing low-redundancy, high-fidelity feature input for subsequent cross-modal fusion. Through the collaboration of the modality-specific encoder and the adaptive bottleneck layer, the spatial distribution features of radar, the spatiotemporal evolution features of satellites, the motion features of UAVs, and the semantic features of text maintain semantic consistency after dimensionality reduction, significantly improving the operational correlation analysis capability under complex sea conditions. The adaptive bottleneck layer compresses the single-modal feature dimension from 1024 dimensions to 64 dimensions, reducing the computation of cross-modal fusion, compressing the time consumption of operational analysis, and reducing the energy consumption of model inference, providing lightweight, low-power feature processing capabilities for maritime edge computing devices.This invention constructs a multimodal feature intelligent compression and semantic alignment system for maritime vessel combat data through feature decoupling extraction using a modal-specific encoder and dimensionality-reduction and fidelity compression with an adaptive bottleneck layer. This system enables combat systems to achieve high-fidelity cross-modal feature extraction, low power consumption across the entire computational chain, and high adaptability to complex scenarios. It promotes a paradigm shift in maritime emergency response from "coarse-grained feature fusion" to "fine-grained semantic verification," providing a zero-information-loss, low-computational-overhead intelligent feature foundation for battlefield situation assessment and dynamic decision generation.

[0085] As shown in Figure 1, 13, based on the dimensionality-reduced feature vectors of each modality, a multi-level attention bottleneck unit is constructed in the large model. Each modality feature vector is assigned to its corresponding level for attention weight calculation, generating updated bottleneck unit parameters, including:

[0086] 131. Deploy a hierarchical attention mechanism in the bottleneck unit. The bottom attention layer calculates the correlation weight of features within the modality, the middle attention layer calculates the complementarity weight of features across modalities, and the top attention layer calculates the contribution weight of global features.

[0087] 132. The parameters for generating and updating the bottleneck unit include:

[0088] A soft routing algorithm is used to calculate the allocation probability of each modality feature vector to different attention layers, and the feature vectors are weighted and recombined according to the probability distribution.

[0089] In each attention layer, the recombined feature vector is concatenated with the learnable positional encoding, and the state parameters of the bottleneck unit are updated through residual connections.

[0090] In this embodiment of the invention, the multi-level attention bottleneck unit realizes the progressive fusion of multimodal features through a hierarchical attention mechanism. The bottom layer strengthens the association of key features within a modality through a self-attention mechanism, the middle layer captures semantic complementary information between different modalities through a cross-attention mechanism, and the top layer evaluates the global importance of features through a task-oriented attention mechanism. Features are dynamically allocated to different levels through a soft routing algorithm. Combined with positional encoding and residual connections, the problem of modal heterogeneity and semantic alignment in multimodal feature fusion is effectively solved.

[0091] Low-level attention, using Calculate the correlation weights between features within a single modality, where, For relevance weights, , These are the query and key matrices for modality m, respectively. The dimension of the key vector;

[0092] Mid-level attention, using Calculate the complementary weights between different modal features, where, As complementary weights, The query matrix is ​​from mode m. Let n be the bond matrix from mode n. The dimension of the key vector;

[0093] Top-level attention, using The evaluation features contribute to the final task, among which, As for contribution level, The weight matrix is ​​related to the task. The characteristic representation of mode m;

[0094] Using a soft routing algorithm Calculate the probability of assigning features to different levels, where, To assign probabilities, For learnable parameter matrix, For bias terms, For the target level (bottom / middle / top). As a characteristic;

[0095] Based on route probability weighted fusion Calculate features at different levels, where, For features at different levels, L is the total number of levels. To assign probabilities, As a feature, For target level;

[0096] use Location encoding fusion is performed to enhance the temporal or spatial location awareness of features, whereby... Let m be the learnable positional encoding matrix for mode m. For features at different levels, Features after fusion;

[0097] use Perform residual updates while preserving original feature information to prevent gradient vanishing. The parameters are the parameters after the residual update, and FFN is a feedforward neural network. For features at different levels, These are the features after fusion.

[0098] As shown in Figure 1, 14, cross-modal information interaction is performed in the updated bottleneck unit. Dynamic fusion of feature vectors from different modalities is achieved through feature concatenation and attention mechanisms, generating multimodal fusion information containing spatiotemporal correlations, including:

[0099] 141. Construct a dynamic graph structure with modal features as nodes and cross-modal associations as edges, and use a graph attention network to iteratively update the message passing paths between nodes;

[0100] 142. In each graph update iteration, nonlinear spatial dependencies between cross-modal features are captured by deformable convolutional kernels, and feature interaction patterns of different subspaces are fused using a multi-head attention mechanism.

[0101] 143. Perform dual-path aggregation of max pooling and average pooling on the node features after multiple rounds of graph updates to generate multimodal fusion information that retains both local details and global context.

[0102] In this embodiment of the invention, the temporal and spatial correlations between cross-modal features are captured through a node-edge dynamic graph, solving the problem of heterogeneous modal feature alignment; deformable convolutional kernels adaptively capture the nonlinear spatial interactions between cross-modal features, breaking through the fixed receptive field limitation of traditional convolution; a multi-head attention mechanism fuses feature interaction patterns from different subspaces, enhancing the diversity of feature representations; max pooling preserves local salient features, and average pooling extracts global context, generating fused features that combine detail and globality;

[0103] Each modal feature is used as a node, and cross-modal associations are used as edges to construct a dynamic graph structure, which is represented as follows: Where G(t) is the dynamic graph structure, V is the set of modal nodes, and M is the number of modes. Let i be the set of edges at the t-th iteration, and let i and j be the indices of the nodes and edges, respectively.

[0104] Using graph attention networks, Iteratively update node features, where... The features of node i at the t-th iteration are... Let be the feature of node i at the (t+1)th iteration, σ be the activation function, i and j be the indices of the node and edge respectively, N be the total number of sets, and W be the weight matrix. The edge weights are calculated using scaled dot product attention: ,in, Let d be the learnable parameter matrix, and d be the feature dimension. The features of node i at the t-th iteration;

[0105] This approach represents features from different modalities, such as images, text, and speech, as graph nodes. It directly characterizes the intermodal relationships through edges between nodes, overcoming the limitations of implicit feature concatenation in traditional methods. A Graph Attention Network (GAT) is used to iteratively update the attention weights (i.e., edge strength) between nodes, enabling the model to dynamically focus on key modal relationships based on task requirements (e.g., prioritizing the interaction between the target object in the image and the question text in a visual question-answering task). Visualizing the attention weights allows for intuitive analysis of the model's dependence on different modal relationships. Dynamically adjusting message passing paths effectively mitigates modality loss or noise interference.

[0106] Adaptive adjustment via deformable convolution kernels, using The nonlinear spatial dependencies are captured, and then the feature interaction patterns of different subspaces are fused through a multi-head attention mechanism. It is nonlinear spatially dependent. The coordinates of the center point, This is the offset of a regular convolution kernel. These are learnable offsets, generated from feature predictions. Convolutional kernel weights The features of node i at the t-th iteration;

[0107] In cross-modal feature alignment, deformable convolution dynamically adapts to the spatial distribution differences of features across different modalities by learning the offsets of feature points. Compared to the fixed receptive field of traditional convolution, deformable convolution can capture more complex inter-modal geometric transformations, enhancing the flexibility of feature alignment. By processing the interactions of different modal subspaces in parallel through multiple attention heads, the representation bottleneck of a single feature space is avoided. Each attention head independently calculates weights and fuses features, ultimately capturing high-order nonlinear relationships between modalities through concatenation or weighted aggregation. The combination of nonlinear spatial dependence and multi-subspace fusion enables the model to represent complex cross-modal associations that are difficult to capture by traditional methods. The decoupling of different subspaces by multi-head attention can reduce the impact of single-modal noise on the overall interaction.

[0108] By generating fused features that combine local details and global context through dual-path pooling, the following methods are used: Max pooling and average pooling are performed, followed by feature fusion. The node features after the final iteration. Max pooling features are used to preserve locally significant features. For average pooling features, global context is extracted, where V is the set of modal nodes. In the node features after multiple rounds of graph updates, max pooling retains the maximum response value in each modal channel (such as the edge of the target object in the image, keywords in the text), strengthening local discriminative information. Noise suppression: by taking the maximum value operation, low-response noise features are naturally filtered out. Average pooling calculates the mean of all modal features to generate a globally consistent contextual representation, avoiding overly one-sided local features. The mean operation alleviates the interference of outliers in local features. The local sharpening features of max pooling are concatenated or weighted with the global smoothing features of average pooling to generate a multimodal fusion representation that combines details and context.

[0109] As shown in Figure 1, 15, the multimodal fusion information is input into the classifier of the large model for battlefield situation assessment, and a dynamic strike decision scheme is generated based on the assessment results, including:

[0110] 151. Based on the battlefield situation assessment results, a multi-objective optimization algorithm is used to solve the optimal fire allocation scheme under the constraints of combat resources;

[0111] 152. Construct a route planning model, integrate battlefield environment data to construct a four-dimensional threat field, and generate a flight path through an ant colony algorithm. The four-dimensional threat field includes air threats, surface threats, underwater threats, and electromagnetic threats.

[0112] 153. Based on the ship type and the status of combat resource inventory, use knowledge graph reasoning technology to match the optimal combination of combat equipment and generate a phased mission execution instruction set.

[0113] In this embodiment of the invention, considering factors such as ammunition inventory, weapon range, and target threat level, firepower is dynamically allocated under resource constraints. For example, priority is given to striking high-value stealth targets, and the Pareto optimal solution algorithm is used to balance the conflict between "strike efficiency" and "resource consumption." Data from radar (air threats), sonar (underwater threats), and electronic reconnaissance (electromagnetic threats) are integrated to construct a dynamically updated four-dimensional threat field model. By improving the ant colony algorithm and introducing reinforcement learning heuristics, the optimal penetration path for ships is planned. Based on knowledge graph reasoning, the best countermeasure equipment is automatically matched according to the target type, and a phased instruction set is generated to reduce the cognitive load on commanders. Based on multimodal fusion information, the optimal combat decision is generated through intelligent algorithms, realizing a closed loop from battlefield perception to combat execution.

[0114] As shown in Figure 2, a situational awareness system 20 based on a large model for integrated land, sea, and air maritime combat scenarios includes:

[0115] The acquisition module 21 is used to acquire multi-source combat data from land, sea and air and perform spatiotemporal alignment processing to obtain data after each mode alignment. The multi-source combat data from land, sea and air includes radar detection data of surface ships, infrared image data of air fighter jets and communication command data of land command system.

[0116] Processing module 22 is used to input the aligned data of each modality into the multimodal encoder of the large model for parallel feature extraction, and to reduce the dimensionality of the extracted features through the adaptive bottleneck layer of the large model to obtain the dimensionality-reduced feature vectors of each modality; based on the dimensionality-reduced feature vectors of each modality, a multi-level attention bottleneck unit is constructed in the large model, and each modal feature vector is assigned to the corresponding level for attention weight calculation to generate updated bottleneck unit parameters; cross-modal information interaction is performed in the updated bottleneck unit, and the dynamic fusion of feature vectors of different modalities is realized through feature concatenation and attention mechanism to generate multimodal fusion information containing spatiotemporal correlation; the multimodal fusion information is input into the classifier of the large model for battlefield situation assessment, and a dynamic strike decision scheme is generated based on the assessment results.

[0117] Figure 3 is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention.

[0118] As shown in Figure 3, the electronic device may include a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions from the memory 630 to execute a situational awareness method for integrated land, sea, and air combat scenarios based on a large model.

[0119] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the situational awareness method for integrated land, sea and air combat scenarios based on a large model provided by the above methods.

[0121] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the situational awareness method for integrated land, sea and air combat scenarios based on a large model provided by the above methods.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A situational awareness method for integrated land, sea, and air maritime combat scenarios based on a large model, characterized in that... include: The process involves acquiring multi-source combat data from land, sea, and air sources and performing spatiotemporal alignment to obtain modally aligned data. This multi-source data includes radar detection data from surface ships, infrared image data from airborne aircraft, and communication command data from land-based command systems. The modally aligned data is then input into a large-scale multimodal encoder for parallel feature extraction. The extracted features are then dimensionality-reduced using an adaptive bottleneck layer of the large-scale model to obtain dimensionality-reduced feature vectors for each modality. This includes: constructing a radar point cloud feature encoder, using a 3D sparse convolutional network to extract the spatial distribution features of obstacles around the ship; constructing a satellite remote sensing feature encoder, using a multispectral attention network to extract the spatiotemporal evolution features of the sea surface temperature field and ocean current vector field, including building a spatiotemporal dual-channel attention module based on multispectral satellite data, and extracting the spatiotemporal coupled evolution features of the sea surface temperature field and ocean current vector field through the synergy of spectral and temporal attention; and constructing a UAV video feature encoder, using spatiotemporal transformation... The system extracts continuous inter-frame motion features of abnormal ship attitude behavior using a network; constructs a text semantic feature encoder and uses a pre-trained language model to extract keyword entities and sentiment features from crew report text; the adaptive bottleneck layer reduces the dimensionality of the extracted features by mapping each modality feature to a low-dimensional subspace using a learnable projection matrix and dynamically adjusting the compression ratio of each modality feature using a gating mechanism to generate a dimensionality-reduced feature vector that retains key information; based on the dimensionality-reduced feature vectors of each modality, a multi-level attention bottleneck unit is constructed in the large model, and each modality feature vector is assigned to the corresponding level for attention weight calculation to generate updated bottleneck unit parameters, including: deploying a hierarchical attention mechanism in the bottleneck unit, with the bottom attention layer calculating the intra-modality feature correlation weight, the middle attention layer calculating the cross-modality feature complementarity weight, and the top attention layer calculating the global feature contribution weight; the generation of updated bottleneck unit parameters includes: using a soft routing algorithm. Calculate the probability of assigning features to different levels, where, To assign probabilities, For learnable parameter matrix, For bias terms, The target hierarchy includes bottom, middle, and top levels. For features, the allocation probability of each modal feature vector to different levels of attention layers is calculated, and the feature vectors are weighted and recombined according to the probability distribution. In each level of attention layer, the recombined feature vector is concatenated with the learnable position code, and the state parameters of the bottleneck unit are updated through residual connection. Cross-modal information interaction is performed in the updated bottleneck unit, and the dynamic fusion of different modal feature vectors is realized through feature concatenation and attention mechanism to generate multimodal fusion information containing spatiotemporal correlation. The multimodal fusion information is input into the classifier of the large model for battlefield situation assessment, and a dynamic strike decision scheme is generated according to the assessment results, including: based on the battlefield situation assessment results, a multi-objective optimization algorithm is used to solve the optimal fire allocation scheme under the constraints of combat resources; a route planning model is constructed, a four-dimensional threat field is constructed by integrating battlefield environment data, and a navigation path is generated by ant colony algorithm. The four-dimensional threat field includes air threats, surface threats, underwater threats and electromagnetic threats; according to the ship type and combat resource inventory status, the optimal combat equipment combination is matched by knowledge graph reasoning technology, and a phased mission execution instruction set is generated.

2. The situational awareness method for integrated land, sea, and air combat scenarios based on a large model as described in claim 1, characterized in that, The process involves acquiring multi-source combat data from land, sea, and air sources and performing spatiotemporal alignment processing to obtain modally aligned data. This multi-source combat data includes radar detection data from surface ships, infrared image data from airborne aircraft, and communication command data from land-based command systems. The process includes: standardizing and normalizing the multi-source combat data from different sources using a large-scale model preprocessing module to generate preprocessed data with a unified data format; performing time alignment on the preprocessed data, matching timestamps from different data sources based on a time-series dynamic time warping algorithm to generate time-synchronized intermediate data; performing spatial alignment on the intermediate data, transforming the coordinate system of the multi-source combat data from land, sea, and air to a preset global spatial reference system to generate spatially aligned intermediate data; and performing inter-modal correlation matching between the time-synchronized intermediate data and the spatially aligned intermediate data to generate modally aligned data with spatiotemporal consistency constraints.

3. The situational awareness method for integrated land, sea, and air combat scenarios based on a large model as described in claim 2, characterized in that, The intermediate data after time synchronization and the intermediate data after spatial alignment are matched for intermodal correlation to generate modally aligned data with spatiotemporal consistency constraints. This includes: extracting radar pulse time series, satellite scan time series, and UAV video frame time series from preprocessed data to construct a multi-source heterogeneous time series set; using a dynamic time warping algorithm to calculate the cumulative distance matrix of any two time series in the multi-source heterogeneous time series set, and determining the optimal alignment path through a backtracking mechanism; performing nonlinear interpolation on the time series according to the optimal alignment path to eliminate the timing deviation caused by sampling frequency differences, and generating a time-aligned synchronized sequence; the spatial alignment operation includes: establishing a global spatial coordinate system centered on the ship in the maritime combat scenario, and calculating the transformation matrix from radar polar coordinates, satellite geographic coordinates, and UAV visual coordinates to the global coordinate system; mapping the spatial location information of different data sources to the same spatial reference system through the coordinate transformation matrix, and using a Kalman filter algorithm to eliminate spatial measurement noise, generating spatially aligned high-confidence location data.

4. The situational awareness method for integrated land, sea, and air combat scenarios based on a large model as described in claim 3, characterized in that, Cross-modal information interaction is performed in the updated bottleneck unit. Dynamic fusion of feature vectors from different modalities is achieved through feature concatenation and attention mechanisms to generate multimodal fusion information containing spatiotemporal correlations. This includes: constructing a dynamic graph structure with modal features as nodes and cross-modal correlations as edges; using a graph attention network to iteratively update the message passing paths between nodes; capturing the nonlinear spatial dependencies between cross-modal features through deformable convolutional kernels in each graph update iteration, and using a multi-head attention mechanism to fuse feature interaction patterns from different subspaces; and performing dual-path aggregation of max pooling and average pooling on the node features after multiple rounds of graph updates to generate multimodal fusion information that simultaneously preserves local details and global context.

5. A situational awareness system for integrated land, sea, and air maritime combat scenarios based on a large model, characterized in that, include: The acquisition module acquires multi-source combat data from land, sea, and air sources and performs spatiotemporal alignment processing to obtain data aligned to each modality. This multi-source combat data includes radar detection data from surface ships, infrared image data from airborne aircraft, and communication command data from land-based command systems. The processing module inputs the aligned data into a large-scale model's multimodal encoder for parallel feature extraction. It then uses an adaptive bottleneck layer of the large-scale model to reduce the dimensionality of the extracted features, obtaining dimensionality-reduced feature vectors for each modality. This includes: constructing a radar point cloud feature encoder, using a 3D sparse convolutional network to extract the spatial distribution features of obstacles around the ship; constructing a satellite remote sensing feature encoder, using a multispectral attention network to extract the spatiotemporal evolution features of the sea surface temperature field and ocean current vector field, including constructing a spatiotemporal dual-channel attention module based on multispectral satellite data, extracting the spatiotemporal coupled evolution features of the sea surface temperature field and ocean current vector field through the synergy of spectral and temporal attention; and constructing a UAV video feature encoder. The method employs a spatiotemporal transformer network to extract continuous inter-frame motion features of abnormal ship attitude behavior; constructs a text semantic feature encoder and uses a pre-trained language model to extract keyword entities and sentiment features from crew report text; the adaptive bottleneck layer reduces the dimensionality of the extracted features by mapping each modal feature to a low-dimensional subspace using a learnable projection matrix and dynamically adjusting the compression ratio of each modal feature using a gating mechanism to generate a dimensionality-reduced feature vector that retains key information; based on the dimensionality-reduced feature vectors of each modality, a multi-level attention bottleneck unit is constructed in the large model, and each modal feature vector is assigned to a corresponding level for attention weight calculation to generate updated bottleneck unit parameters, including: deploying a hierarchical attention mechanism in the bottleneck unit, with the bottom attention layer calculating the intra-modal feature correlation weight, the middle attention layer calculating the cross-modal feature complementarity weight, and the top attention layer calculating the global feature contribution weight; the generation of updated bottleneck unit parameters includes: using a soft routing algorithm. Calculate the probability of assigning features to different levels, where, To assign probabilities, For learnable parameter matrix, For bias terms, The target hierarchy includes bottom, middle, and top levels. For features, the allocation probability of each modal feature vector to different levels of attention layers is calculated, and the feature vectors are weighted and recombined according to the probability distribution. In each level of attention layer, the recombined feature vector is concatenated with the learnable position code, and the state parameters of the bottleneck unit are updated through residual connection. Cross-modal information interaction is performed in the updated bottleneck unit, and the dynamic fusion of different modal feature vectors is realized through feature concatenation and attention mechanism to generate multimodal fusion information containing spatiotemporal correlation. The multimodal fusion information is input into the classifier of the large model for battlefield situation assessment, and a dynamic strike decision scheme is generated according to the assessment results, including: based on the battlefield situation assessment results, a multi-objective optimization algorithm is used to solve the optimal fire allocation scheme under the constraints of combat resources; a route planning model is constructed, a four-dimensional threat field is constructed by integrating battlefield environment data, and a navigation path is generated by ant colony algorithm. The four-dimensional threat field includes air threats, surface threats, underwater threats and electromagnetic threats; according to the ship type and combat resource inventory status, the optimal combat equipment combination is matched by knowledge graph reasoning technology, and a phased mission execution instruction set is generated.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the situational awareness method for integrated maritime combat scenarios based on a large model, as described in any one of claims 1 to 4.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the situational awareness method for integrated maritime combat scenarios based on a large model, as described in any one of claims 1 to 4.

Images