Land-sea-air-space holographic perception and collaborative decision-making system based on multi-mode edge intelligence

Abstract

The invention discloses a land-sea-air-space holographic sensing and collaborative decision-making system based on multi-mode edge intelligence. The system comprises a sensing layer which is used for carrying out three-dimensional monitoring on a ship driving process, the sea, the sky, a shore base and a space formed by the ship driving process, the sea, the sky, the shore base and the sea, the sky and the shore base; the network layer is used for constructing a global communication basis, preprocessing monitoring data, judging whether the ship has an abnormal condition or not, and realizing data transmission of the ocean, the sky, a shore base and a space formed by the ocean, the sky and the shore base; and the application layer is used for judging whether an abnormal condition exists in the ship driving process or not based on the data of the ocean, the sky, the shore base and the space formed by the ocean, the sky and the shore base transmitted by the network layer, performing collision early warning decision based on the abnormal condition, and performing dynamic channel capacity prediction. And navigation guidance and route tracking in extreme weather are carried out based on environmental risk early warning, underwater obstacle collision is avoided, and early warning information and decision information are fed back to a sensing layer.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of Internet of Things, and relates to a land-sea-air-space holographic perception and collaborative decision-making system based on multi-modal edge intelligence. BACKGROUND

[0002] In the current situation of deep global economic integration and rapid rise of marine strategic value, as the backbone of international trade, maritime transportation carries 90% of the world's trade volume, and its safety and efficiency are related to the world's economic lifeline. However, the traditional maritime management system has many drawbacks, and the economic loss caused by maritime accidents each year exceeds 12 billion US dollars, highlighting the urgency of reform.

[0003] Global trade prosperity has led to an increase in maritime transportation, a significant increase in the number of ships and navigation density, a complex traffic environment, and an increase in navigation risks due to frequent extreme weather. The traditional maritime management system relies on limited monitoring means and cannot fully cover the sea area and real-time dynamic information.

[0004] The core problems are as follows: data islands are serious, the data of various monitoring equipment and systems is isolated, and the data of space-based satellites, air-based aircraft, sea-based ships, and shore-based radars cannot be shared and integrated, resulting in a lack of comprehensive and accurate data support for maritime decision-making; collaborative response is slow, data transmission and processing are inefficient, information between departments is not smooth, and there is a long delay in collaborative rescue in the event of a maritime emergency, which seriously weakens the safety guarantee capability; the dynamic perception accuracy is poor, and the positioning and identification accuracy of traditional monitoring technology in complex sea conditions and various targets is insufficient, making it difficult to accurately track small ships and floating objects and unable to effectively predict ship collision risks.

[0005] The land-sea-air-space integrated information network technology has emerged as the times require, building a four-dimensional perception matrix of "air-sky-sea-land", integrating AIoT, edge computing, and digital twin technology. Space-based navigation and remote sensing integrated satellites, air-based SAR satellites and airborne CSSAR systems, sea-based VTS radar networking and buoy sensors, and shore-based optoelectronic equipment work collaboratively. The network layer solves the problems of data transmission and processing through heterogeneous network fusion and edge computing nodes, laying a solid foundation for maritime information flow and real-time decision-making, and opening up new avenues for tackling maritime safety management challenges. SUMMARY

[0006] To solve the above problems, the technical scheme adopted by the application is: a multi-modal edge intelligent land-sea-air-space holographic perception and collaborative decision-making system, comprising: a perception layer for multi-dimensional three-dimensional monitoring of the ship driving process, the ocean, the sky, the shore, and the space formed by the three; Network layer: based on the perception layer, the data of the ship running process, the ocean, the sky, the shore base and the space composed of the three are fused, the global communication foundation is built, the monitoring data is preprocessed, whether the ship exists abnormal condition is judged, and the data transmission of the ocean, the sky, the shore base and the space composed of the three is realized; Application layer: based on the data of the ocean, the sky, the shore base and the space composed of the three transmitted by the network layer, whether the abnormal condition exists in the ship running process is judged, the ship navigation guidance and route tracking decision are made based on the abnormal condition, the dynamic channel density and capacity prediction are carried out, the navigation guidance and route tracking in extreme weather and the avoidance of underwater obstacle collision are carried out based on the environmental risk early warning, and the above early warning information and decision information are fed back to the perception layer.

[0007] Further: the perception layer comprises: Space-based perception node: ocean and land observation data acquisition, receiving and processing during ship navigation process are carried out; Air-based perception node: the ocean dynamics during ship navigation process are monitored, the time-sensitive targets are tested, and the weather conditions are monitored, high-resolution imaging and monitoring data are provided for ship navigation guidance and route tracking; Sea-based perception node: ocean environment parameters, ship dynamics and heading track during ship navigation process are monitored, and accurate target position data are provided for collision avoidance; Shore-based perception node: used for realizing real-time monitoring and collection of port sea surface wave field, and tracking ship motion trajectory.

[0008] Further: the space-based perception node relies on integrated satellite architecture of navigation and remote sensing, comprising space end, ground end and user end; The space end: integrating high-resolution multi-mode remote sensing, two-way Internet of Things communication and satellite-based navigation enhancement function, and single satellite: providing high-resolution ocean and land observation data, assisting in identifying potential collision risk area, realizing real-time data transmission between ship and satellite, ensuring timely transmission of early warning information, and providing high-precision positioning service; Ground end: assisting in identifying potential navigation collision risk area based on the ocean and land observation data transmitted by the space end, receiving, processing and distributing; User end: based on the data distributed by the ground end, realizing service access in ship navigation, and guaranteeing navigation monitoring and emergency communication.

[0009] Further: the sea-based perception node comprises: Buoy sensor network: used for collecting ocean environment parameters, VTS radar networking: used for monitoring ship dynamics; Automatic Identification System (AIS): for mutual identification between ships during navigation; Electronic Chart Engine (ECE): for automatic route design, course and track monitoring, automatic storage of ship's track, historical voyage re-demonstration, and automatic navigation warning. Further, the space-based perception node comprises: SAR satellite: all-weather imaging capability penetrates cloud and night restrictions, continuously monitors ocean dynamics; CSSAR system: for time-sensitive target monitoring, timely discovery and early warning of potential collision risks; Atmospheric detection equipment: including weather radar for real-time monitoring of severe sea conditions; Atmospheric composition analyzer detects air pollution levels.

[0010] Further, the network layer comprises: Fusion of heterogeneous networks: build a communication foundation for the sea, sky, and shore, and the space formed by the three, the fusion of heterogeneous networks includes: 5G / 4G network: realize real-time data transmission of ports and near-sea channels, support high-definition video backhaul and dense sensor data access, and provide low-latency shore-based support for collision warning; Low-orbit satellite constellation: provides high-bandwidth, low-latency communication services for ocean-going ships, ensures real-time data interaction between ship terminals, VTS centers, and buoy nodes, and ensures the safety of ocean navigation; Shore-based backbone network: uses microwave relay and optical fiber network to build high-speed transmission links between shore-based data centers and remote perception nodes, supporting data aggregation and decision-making instruction distribution; Data processing module: including preprocessing module, anomaly detection module and decision module; The preprocessing module comprises: Real-time compression module: for real-time compression of raw data collected by radar, cameras, and hydrological sensors using differential encoding, run-length encoding, and adaptive quantization lightweight real-time compression algorithms to meet real-time requirements; Adjustment module: for dynamically adjusting the compression ratio according to network conditions to prioritize transmission efficiency of collision warning instructions; Anomaly detection module: for establishing a ship AIS trajectory time series feature analysis and prediction model based on a combination of long short-term memory network (LSTM) and graph attention network (GAT); The ship AIS trajectory time series feature analysis and prediction model uses an encoder-decoder structure, which includes an encoder module, the encoder module includes a history encoder and an interaction encoder, which respectively encode the spatial and temporal interactions between ships; The historical encoder consists of stacked LSTMs that extract motion trend sequence features of each ship from historical trajectories in the time dimension; The interactive encoder consists of a GAT layer combined with an LSTM. The GAT models the interactions between different ships in the sea area in the spatial dimension, enhancing the interaction between the target ship and its neighboring ships, and weakening the interaction between ships with low correlation. At the same time, considering the temporal continuity between ship spatial interactions, an additional LSTM is used to capture the temporal continuity of spatial interactions, resulting in spatial interaction features with temporal continuity. Then, the two high-level features extracted by the historical encoder and the interactive encoder are fused to form an intermediate state vector, which is input into the decoder module composed of LSTM to predict the ship AIS temporal trajectory information in complex scenarios. If abnormal behavior is detected in the trajectory, the abnormal data is marked and a local warning is triggered. By combining hydrological and meteorological sensor data and using threshold comparisons to predict extreme weather, warnings are sent to nearby vessels and navigation signals are adjusted. Decision module: Based on the abnormal ship behavior transmitted by the anomaly detection module and the predicted extreme weather, it makes collision avoidance decisions; the process of making collision avoidance decisions based on the abnormal ship behavior transmitted by the anomaly detection module and the predicted extreme weather is as follows: Based on the ship dynamic monitoring model and the waterway situation simulation system, the abnormal ship behavior acquired by the anomaly detection module is integrated with real-time hydrological information and radar / AIS data to construct a three-dimensional dynamic map of the waterway on the digital twin platform. The latest navigation signals are broadcast to the ship using the shipborne ad hoc network for ship navigation guidance and route tracking, guiding the ship to bypass dangerous areas or follow historical routes to assist in navigation risk decision-making. In ship emergency situations, emergency events are determined according to federated learning and emergency response mechanisms, and then transmitted encryptedly through a heterogeneous network fusion system. Finally, the ship terminal executes emergency berthing and directly controls emergency equipment to achieve rapid response.

[0011] Furthermore: the network layer also includes a heterogeneous data network, which includes: Nearshore transmission module: It adopts 5G network to transmit high-definition monitoring video of the port in real time, supports the VTS center to monitor the berthing and departure process of ships in a refined manner, and LoRa network connects sensors in mountainous and remote sea areas to realize low-cost monitoring data transmission. Ocean-going transmission module: Employs BeiDou short message service to ensure the transmission of emergency collision avoidance commands when satellite communication is interrupted, thus ensuring the minimum safe communication requirements of ocean-going vessels; Microwave relay self-organizing network: Constructing a dynamic network of ships and buoys to enable multi-ship collaborative collision avoidance and local data sharing; Space-air-ground-sea transmission module: using low-orbit satellite Internet of Things to realize wide-area coverage communication between unmanned aerial vehicles, high-altitude platforms and satellites for large-scale maritime monitoring and emergency response.

[0012] Further, the application layer comprises: Multi-source data fusion architecture module: eliminate data islands by building an integrated space-air-ground-sea perception network; Achieve nanosecond-level synchronization through Beidou satellite and atomic clock to realize space-time reference alignment; Machine learning model real-time correction of sea wave / electromagnetic interference bias to realize heterogeneous data calibration; Reduce ship positioning error through satellite image and radar echo superposition analysis; Increase target recognition rate through underwater acoustic signal and AIS data cross-validation.

[0013] Further, the application layer further comprises a data hierarchical model module, The data hierarchical model module comprises: Data operation layer: real-time collection of raw data, aggregation of multi-source raw data from space-based, air-based, sea-based and land-based platforms; Data warehouse layer: including a ship trajectory cube module, constructing a multi-dimensional model according to time, space and index for ship parallel analysis; Marine environment data cube module: minute-level sampling according to time, space and index covering major global shipping lanes; Sensor failure prediction in device health assessment model library: through the establishment of a system device self-diagnosis model, the condition of each sensor node is monitored, the condition of the node is analyzed based on signal quality, and combined with the self-diagnosis knowledge base in the database module, the historical fault cases and expert rule base are used to perform pattern matching and root cause analysis on abnormal patterns, generate a diagnosis report containing operation suggestions, and send it to the self-adjusting end for effective predictive maintenance of the system. The system is protected and isolated by corresponding protection measures against malicious attacks from the outside world, and the gateway traffic is monitored; Network node health index: evaluate communication link stability and edge computing node load rate; Data application layer comprises: Real-time traffic control application module, through the ship abnormal behavior detection model input ship trajectory cube and AIS historical data, output yaw warning, overspeed alarm and AIS silence detection probability, support real-time decision of collision warning system, dynamic channel capacity prediction combined with VTS radar ship density and weather cube data, predict the channel capacity of different time periods, and optimize ship traffic flow organization; The environmental risk early warning application module: inputting SAR image target features and weather radar precipitation data through a marine emergency response decision tree, outputting ship avoidance path suggestions and search and rescue priority ranking, supporting collision early warning in extreme weather, analyzing acoustic monitoring array signal features and historical submarine activity patterns through underwater target threat evaluation, generating a threat level heat map, and avoiding underwater obstacle collision; The network optimization application module: according to 5G base station load and satellite link state, outputting a multi-path optimization scheme to ensure real-time transmission of collision early warning instructions, predicting port camera video traffic and buoy sensor data volume through an edge computing resource scheduling model, allocating edge node computing power, and ensuring data preprocessing efficiency.

[0014] The land-sea-air-space holographic perception and collaborative decision-making system based on multi-modal edge intelligence has the following advantages: Focusing on the Internet of Things driven marine land-sea-air-space information detection technology system, a comprehensive intelligent perception and early warning system is constructed. By integrating space-based, air-based, sea-based and shore-based perception nodes, using advanced communication and edge computing technology, global marine target monitoring, efficient data flow and intelligent processing are realized, and a "perception - decision - action" closed loop is formed. Especially for ship collision risk, accurate and real-time early warning is provided, which promotes the transformation of maritime management to intelligent, accurate and real-time, and contributes to the innovation of marine safety protection and management efficiency.

[0015] The land-sea-air-space integrated ship collision early warning system constructed in the application realizes global perception, intelligent decision-making and real-time response by integrating satellite, radar, buoy and other multi-source sensors combined with edge computing technology. The system has the following advantages: accurate perception - four-dimensional cooperation of space-based, air-based, sea-based and shore-based, dynamic calibration of data deviation, and improved ship positioning and environmental monitoring accuracy. Efficient cooperation - heterogeneous network fusion of 5G / satellite / microwave communication, real-time data processing of edge nodes, support for local decision-making and emergency response. Intelligent application - hierarchical data model driven traffic control, environmental warning and network optimization, three-dimensional situation map and API interface to improve interaction efficiency. Value potential - significantly reduce collision risk, optimize channel traffic capacity, promote the intelligent transformation of maritime management, and can be extended to environmental monitoring and emergency command in the future. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiment or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without creative labor.

[0017] Figure 1is a system architecture schematic diagram of the present application; Figure 2 is a technology roadmap of the present application; Figure 3 is a structure schematic diagram of the system of the present application. DETAILED DESCRIPTION

[0018] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict, and the present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0019] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be described clearly and completely below in combination with the drawings of the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. The description of the at least one exemplary embodiment is actually only illustrative, but not as any limitation on the present application and its application or use. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0020] Figure 1 is a system architecture schematic diagram of the present application; The land-sea-air-space holographic perception and collaborative decision-making system based on multi-modal edge intelligence comprises: The perception layer is used for multi-dimensional three-dimensional monitoring of the space formed by the ship running process, the ocean, the sky, the shore base and the three thereof; The network layer fuses the data monitored by the perception layer on the space formed by the ship running process, the ocean, the sky, the shore base and the three thereof, constructs a global communication foundation, pre-processes the monitoring data, judges whether the ship has an abnormal condition, and realizes data transmission of the space formed by the ocean, the sky, the shore base and the three thereof; The application layer judges whether there is an abnormal condition in the ship running process based on the data of the space formed by the ocean, the sky, the shore base and the three thereof transmitted by the network layer, and makes navigation guidance and route tracking decisions based on the abnormal condition, dynamically predicts channel density and capacity, and based on environmental risk early warning, makes ship navigation guidance and route tracking in extreme weather and avoids underwater obstacle collision, and feeds back the above early warning information and decision-making information to the perception layer.

[0021] Figure 2 is a technology roadmap of the present application; Figure 3 is a structure schematic diagram of the system of the present application. The perception layer comprises: Space-based perception node: collect, receive and process ocean and land observation data during ship navigation; Air-based perception node: monitor ocean dynamics during ship navigation, test time-sensitive targets and monitor weather conditions to provide high-resolution imaging and monitoring data for ship navigation guidance and route tracking; Sea-based perception node: monitor ocean environmental parameters, ship dynamics and heading and track during ship navigation to provide accurate target position data for collision avoidance; Shore-based perception node: used to realize real-time monitoring and collection of port sea surface wave field conditions and tracking of ship motion trajectory.

[0022] The space-based perception node relies on integrated satellite architecture, including space end, ground end and user end; The space end: integrates high-resolution multi-mode remote sensing, two-way Internet of Things communication and satellite-based navigation enhancement functions, and a single satellite has multi-source service capability: provides high-resolution ocean and land observation data, assists in identifying potential collision risk areas, realizes real-time data transmission between ships and satellites, ensures timely delivery of warning information, and provides high-precision positioning services; The ground end: receives, processes and distributes ocean and land observation data transmitted by the space end to assist in identifying potential collision risk areas during navigation; The user end: based on the data distributed by the ground end, realizes service access during ship navigation, and ensures navigation monitoring and emergency communication; The ground end and user end form a space-ground cooperation; The development mode: constellation separate integration (such as PNTRC system), multi-constellation function connection, AI large model (various neural network models, machine learning models) optimization of shipping management. Satellite platform integration, single satellite integration of multi-source services, and strengthening of emergency communication guarantee. Deep integration mode, standardized inter-satellite interface + computing power sharing + integrated navigation and positioning signals, improve supervision efficiency.

[0023] Core advantage: data interface unified to realize seamless transfer between ship-satellite-shore; open and shared computing power improves supervision efficiency Navigation and positioning signal fusion makes the positioning error <0.1 meters, improves the accuracy of supervision and emergency response.

[0024] The air-based perception node includes: all-time high-precision monitoring, providing high-resolution imaging and monitoring data for ship navigation guidance and route tracking.

[0025] SAR satellite: all-weather imaging capability penetrates through clouds and night restrictions, sub-meter resolution can clearly identify small targets, and continuously monitors ocean dynamics. Monitor multi-band and multi-polarization technology, C-band specializes in ocean monitoring, providing multi-source data support for collision avoidance; CSSAR system: Circular trajectory scanning realizes 10-second revisit period, suitable for time-sensitive target monitoring. Low-altitude high-resolution spaceborne system, improves emergency response speed, helps to discover and warn potential collision risks in time; Atmospheric detection equipment: Weather radar monitors typhoons, heavy rain and other adverse sea conditions in real time, providing meteorological data support for collision warning. Atmospheric composition analyzer detects air pollution to support emergency command.

[0026] The sea-based sensing node, the near-shore defense barrier, the sea-based sensing node comprises: Buoy sensor network: used for collecting marine environmental parameters, VTS radar networking: used for monitoring the dynamics of ships; Automatic identification system for ships: used for mutual identification between ships during navigation; Electronic chart engine: used for automatic route design, heading and track monitoring, automatic storage of ship's track, historical voyage re-demonstration and automatic navigation alarm. VTS radar networking integrates radar, AIS and other subsystems to realize ship dynamic monitoring and abnormal warning.

[0027] Technical advantages: Precise tracking: The system has leading performance in complex scenarios compared with similar products, improving the accuracy of collision warning.

[0028] Trajectory fusion: Multi-radar + AIS data normalization optimizes target positioning accuracy, providing accurate target position data for collision warning.

[0029] Electronic chart engine: supports panoramic situation awareness and emergency decision-making Buoy sensor network: integrates CTD, ADCP and other equipment to monitor marine environmental parameters such as temperature, salinity, depth and current profile in real time, providing environmental data support for collision warning.

[0030] Shore-based sensing equipment: photoelectric equipment is responsible for high-definition video and laser radar to realize accurate monitoring of the port. High-frequency radar is responsible for monitoring sea surface wind field and tracking target motion trajectory; The network layer comprises: Fusion of heterogeneous networks: build a communication foundation for the sea, sky, shore and the space formed by the three. The fusion of heterogeneous networks comprises: 5G / 4G network: realizes real-time data transmission of ports and near-shore channels, supports high-definition video backhaul and dense sensor data access, provides low-latency shore-based support for collision warning; Low-orbit satellite constellation: provides high-bandwidth, low-latency communication services for ocean-going ships, ensures real-time data interaction between ship terminals, VTS centers and buoy nodes, and ensures the safety of ocean navigation; Coastal backbone network: microwave relay and optical fiber network are used to build high-speed transmission link between coastal data center and remote sensing nodes, supporting data aggregation and decision-making instruction distribution; Data processing module: including preprocessing module, anomaly detection module and decision module; The preprocessing module comprises: Real-time compression module: used for real-time compression of raw data of radar, camera and hydrological sensor by using lightweight compression algorithm; set in port / buoy node; Adjustment module: used for dynamically adjusting compression ratio according to network condition, and preferentially guaranteeing transmission efficiency of collision warning instruction; set in ship terminal node; dynamically adjusting compression ratio according to network condition is a strategy for optimizing data transmission, mainly used for balancing transmission efficiency in the scene of limited network bandwidth or insufficient network throughput, and preferentially guaranteeing transmission efficiency of collision warning instruction; Anomaly detection module: In the port node, the machine learning model of the edge node is used to analyze the ship AIS data in real time, detect abnormal behaviors (such as deviation and overspeed), mark abnormal data and trigger local warning; used for establishing a ship AIS trajectory time series feature analysis prediction model based on the combination of long short-term memory network (LSTM) and graph attention network (GAT); The ship AIS trajectory time series feature analysis prediction model adopts an encoder-decoder structure, and the encoder-decoder comprises an encoder module; The encoder module comprises a history encoder and an interaction encoder, the encoder-decoder structure comprises an encoder module, the encoder module comprises a history encoder and an interaction encoder, and the spatial and temporal interactions between ships are encoded respectively. The history encoder is composed of stacked LSTM, which extracts the motion trend sequence features of each ship from the historical trajectory in the time dimension. The interaction encoder is composed of GAT layer combined with LSTM, GAT models the interaction between different ships in the sea area in the spatial dimension, enhances the interaction between the target ship and the adjacent ships, and weakens the interaction between the ships with less correlation. At the same time, considering the time continuity of the spatial interaction between ships, an additional LSTM is used to capture the time continuity of the spatial interaction, so as to obtain spatial interaction features with time continuity. Then, the two high-level features extracted by the history encoder and the interaction encoder are fused to form an intermediate state vector, which is input into a decoder module composed of LSTM, so as to predict the ship AIS time series trajectory information in a complex scene. If the abnormal behavior of the trajectory is analyzed and the abnormal data is marked and local warning is triggered.

[0031] Buoy node, combined with hydrological and meteorological sensor data, uses threshold comparison to predict extreme weather, sends warning to surrounding ships and adjusts navigation signal; Decision module: based on the abnormal behavior of the ship detected by the anomaly detection module and the predicted extreme weather, make collision avoidance decisions, integrate the abnormal behavior of the ship obtained by the anomaly detection module, combine real-time hydrological information and radar / AIS data, build a three-dimensional dynamic map of the channel on the digital twin platform, use the shipboard ad hoc network to broadcast the latest navigation signal to the ship, guide the ship navigation and track the route, guide the ship to bypass the dangerous area, or track the route according to the historical track, assist in navigation risk decision-making, in the case of ship emergency, according to the federated learning and emergency response mechanism (early warning-instruction-execution closed loop), determine the emergency, then transmit encrypted through the heterogeneous network fusion system (satellite + ad hoc network), finally the ship terminal executes emergency mooring, directly controls the emergency equipment, and realizes rapid response.

[0032] Based on real-time hydrological data, dynamically adjust the position of the buoy or the navigation signal, guide the ship to bypass the dangerous area, in the case of ship emergency, directly control the emergency equipment, realize rapid response, specifically including: The ship terminal directly analyzes the collision avoidance instructions sent by the VTS center, and executes emergency operations (such as turning, slowing down) through automatic rudder or main engine control system without manual intervention.

[0033] The buoy node dynamically adjusts the position of the buoy or the navigation signal based on real-time hydrological data (such as undercurrent, vortex), and guides the ship to bypass the dangerous area.

[0034] Port node: in the case of ship out of control, fire and other emergencies, directly control the tugboat, fireboat and other emergency equipment to realize rapid response.

[0035] The network layer also includes a device health state evaluation model, Fusion sensor state information (such as temperature, voltage), manual measurement data (such as calibration record) and historical monitoring data, predict device failure through random forest or LSTM model, replace aging sensors in advance, avoid monitoring blind area caused by device failure.

[0036] The network layer also includes a heterogeneous data network, which includes: Offshore transmission module: use 5G network to real-time return port high-definition monitoring video, support VTS center to fine monitoring of ship berthing, off-port process, LoRa network connects sensors in mountainous and remote areas, realizes low-cost monitoring data return with low power consumption and long distance characteristics; Far sea transmission module: use Beidou short message to ensure the transmission of emergency collision avoidance instructions when satellite communication is interrupted, ensure the minimum safety communication demand of ocean-going ships; Microwave relay self-organizing network: build a ship-buoy dynamic network for multi-ship collision avoidance and local data sharing; Space-air transmission module: use low-orbit satellite Internet of Things to realize wide-area coverage communication between unmanned aerial vehicles, high-altitude platforms and satellites for large-scale sea monitoring and emergency response.

[0037] The application layer includes: Multi-source data fusion architecture module: eliminate data islands by building an integrated "space-air-ground-sea" perception network; Through Beidou satellite and atomic clock, achieve nanosecond-level synchronization (error <1 μs) to realize space-time reference alignment; Machine learning model real-time correction of sea wave / electromagnetic interference deviation (accuracy improved by 50%+) to realize heterogeneous data calibration; Through satellite image and radar echo superposition analysis, reduce ship positioning error; ship positioning error <0.5 meters; Through cross-validation of underwater acoustic signals and AIS data, increase target recognition rate; target recognition rate >99%; The application layer also includes a data hierarchical model module, The data hierarchical model module includes: Data operation layer (ODS): real-time collection of raw data, aggregation of multi-source raw data from space-based, air-based, sea-based and land-based platforms, and provision of basic monitoring capabilities for collision warning; Data warehouse layer DW: including ship trajectory cube module: build a multi-dimensional model according to time, space (latitude and longitude grid / channel segmentation) and index (speed, heading, AIS information) for 100,000+ ship parallel analysis; Marine environment cube module: minute-level sampling coverage of major global shipping lanes according to time, space and index; Sensor failure prediction in device health assessment model library: based on device state vector and historical failure rate, predict sensor failure risk to avoid monitoring blind spots caused by device failure; Network node health index: through the establishment of system device self-diagnosis model, monitor the status of each sensor node, analyze the node status based on signal quality, and combine the self-diagnosis knowledge base in the database module to use historical failure cases and expert rule base to perform pattern matching and root cause analysis on abnormal patterns, generate a diagnosis report containing operation suggestions, and send it to the self-adjusting end through the human-computer interaction module for effective predictive maintenance of the system. The system can also identify malicious attacks from the outside, monitor gateway traffic, and take appropriate protective measures to protect and isolate the system.

[0038] Data application layer: Real-time traffic control application module: input ship trajectory cube and AIS historical data through ship abnormal behavior detection model, output yaw warning, overspeed alarm and AIS silence detection probability, support real-time decision of collision warning system, dynamic channel capacity prediction combined with VTS radar ship density and weather cube data, predict channel capacity by time period, optimize ship traffic flow organization; Environmental risk warning application module: input SAR image target features and weather radar precipitation data through marine emergency response decision tree, output ship avoidance path suggestion and search and rescue priority ranking, support collision warning in extreme weather, analyze acoustic monitoring array signal features and historical submarine activity patterns through underwater target threat assessment, generate threat level heat map, avoid underwater obstacle collision; Network optimization application module: according to 5G base station load and satellite link state, output multi-path optimization scheme to ensure real-time transmission of collision warning instructions, edge computing resource scheduling model predicts port camera video traffic and buoy sensor data volume, allocates edge node computing power, ensures data preprocessing efficiency.

[0039] System interaction design: improve warning response efficiency User interface: use WebGL technology to build three-dimensional situation map, intuitively display ship position, heading, speed and environmental parameters (such as wave height, flow rate), support real-time monitoring and decision of collision warning.

[0040] API service: RESTful interface supports ship spatio-temporal range filtering and environmental parameter subscription, supports multi-party docking of VTS center, port dispatching system, etc.

[0041] Emergency response process: design automated emergency response chain, from warning trigger, instruction generation to closed-loop process of execution feedback.

[0042] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A multi-modal edge intelligence based land, sea, air, space holographic perception and collaborative decision system, characterized in that: Comprise: The perception layer: for the ship running process, the ocean, the sky, the shore base and the space composed of the three, multi-dimensional three-dimensional monitoring; Network layer: based on the perception layer, the data of the ship running process, the ocean, the sky, the shore base and the space composed of the three are fused, the global communication foundation is constructed, the monitoring data is pretreated, whether the ship exists abnormal condition is judged, and the data transmission of the ocean, the sky, the shore base and the space composed of the three is realized; Application layer: based on the data of the ocean, the sky, the shore base and the space composed of the three transmitted by the network layer, whether the abnormal condition exists in the ship running process is judged, the ship navigation guidance and the route tracking decision are made based on the abnormal condition, the dynamic channel density and the capacity prediction are carried out, the navigation guidance and the route tracking under the extreme weather and the avoidance of underwater obstacle collision are carried out based on the environmental risk early warning, and the above early warning information and decision information are fed back to the perception layer.

2. The multi-modal edge intelligence based air, land, sea, space, and cyberspace holographic perception and collaborative decision system of claim 1, wherein: The perception layer comprises: Space-based perception node: ocean and land observation data collection, receiving and processing during ship navigation process; Air-based perception node: monitoring the ocean dynamics during ship navigation process, testing time-sensitive targets and monitoring weather conditions, providing high-resolution imaging and monitoring data for ship navigation guidance and route tracking; Sea-based perception node: monitoring ocean environmental parameters, ship dynamics and heading track during ship navigation process, providing accurate target position data for collision avoidance; Shore-based perception node: for realizing real-time monitoring and collection of port sea surface wave field conditions and tracking ship motion trajectory.

3. The multimodal edge intelligence based land, sea, air, space holographic perception and collaborative decision system according to claim 2, characterized in that: The space-based perception node relies on integrated satellite architecture, including space end, ground end and user end; The space end: integrates high-resolution multi-mode remote sensing, two-way Internet of Things communication and satellite-based navigation enhancement functions, and single satellite: provides high-resolution ocean and land observation data, assists in identifying potential collision risk areas, realizes real-time data transmission between ships and satellites, ensures timely delivery of early warning information, and provides high-precision positioning services; The ground end: assists in identifying potential navigation collision risk areas based on the ocean and land observation data transmitted by the space end, receives, processes and distributes; User end: based on the data distributed by the ground end, realizes service access in ship navigation, and guarantees navigation monitoring and emergency communication.

4. The multimodal edge intelligence based land, sea, air, space holographic perception and collaborative decision system as claimed in claim 2, wherein: The sea-based perception node comprises: Buoy sensor network: for collecting ocean environmental parameters, VTS radar networking: for monitoring ship dynamics; Automatic ship identification system: for mutual identification between ships during navigation; Electronic chart engine: for automatic route design, heading and track monitoring, automatic storage of ship track, historical voyage re-demonstration and automatic navigation alarm.

5. The multimodal edge intelligence based air, land, sea, space, and cyberspace holographic perception and collaborative decision system as claimed in claim 2, wherein: The air-based perception node comprises: SAR satellite: all-weather imaging capability penetrates through clouds and night restrictions, continuously monitors ocean dynamics; CSSAR system: monitors time-sensitive targets, and timely discovers and warns potential collision risks; Atmospheric detection equipment: including weather radar for monitoring real-time severe sea conditions; Atmospheric composition analyzer detects air pollution level.

6. The multimodal edge intelligence based air, land, sea, space, and cyberspace holographic perception and collaborative decision system, as claimed in claim 1, wherein: The network layer comprises: Fused heterogeneous network: build a communication foundation for the space composed of the ocean, the sky, the shore base and the three, the fused heterogeneous network includes: 5G / 4G network: realize real-time data transmission of the port and the near-sea channel, support high-definition video backhaul and dense sensor data access, and provide low-delay shore base support for collision warning; Low-orbit satellite constellation: provide high-bandwidth, low-delay communication services for ocean-going ships, ensure real-time data interaction between ship terminals and VTS centers and buoy nodes, and ensure the safety of ocean navigation; Shore base backbone network: build high-speed transmission links between shore base data centers and remote sensing nodes using microwave relay and optical fiber networks to support data aggregation and decision-making instruction distribution; Data processing module: including preprocessing module, anomaly detection module and decision module; The preprocessing module includes: Real-time compression module: used for real-time compression of the collected radar, camera and hydrological sensor raw data using differential encoding, run-length encoding and adaptive quantization lightweight real-time compression algorithm to meet the real-time requirement; Adjustment module: used for dynamically adjusting the compression ratio according to the network condition to preferentially guarantee the transmission efficiency of the collision warning instruction; Anomaly detection module: used for establishing a ship AIS trajectory time series feature analysis and prediction model based on a method combining long short-term memory network (LSTM) and graph attention network (GAT); The ship AIS trajectory time series feature analysis and prediction model adopts an encoder-decoder structure, which includes an encoder module, the encoder module includes a history encoder and an interaction encoder, which respectively encode the spatial and temporal interaction between ships; The history encoder is composed of stacked LSTM, which extracts the motion trend sequence features of each ship in the historical trajectory in the time dimension; The interaction encoder is composed of GAT layer combined with LSTM, GAT models the interaction between different ships in the sea area in the spatial dimension, enhances the interaction between the target ship and the adjacent ships, and weakens the interaction between the ships with less correlation; At the same time, considering the time continuity of the spatial interaction between ships, an additional LSTM is used to capture the time continuity of the spatial interaction to obtain spatial interaction features with time continuity; Then, the two high-level features extracted by the history encoder and the interaction encoder are fused to form an intermediate state vector, which is input into a decoder module composed of LSTM to predict the AIS time series trajectory information of the ship in a complex scenario, if the abnormal behavior of the trajectory is analyzed, the abnormal data is marked and local warning is triggered; Combined with the hydrological and meteorological sensor data, the threshold comparison is used to judge and predict extreme weather, and the surrounding ships are sent a warning and the navigation signal is adjusted; Decision module: based on the ship abnormal behavior transmitted by the anomaly detection module and the predicted extreme weather, make collision avoidance decision; The process of making collision avoidance decision based on the ship abnormal behavior transmitted by the anomaly detection module and the predicted extreme weather is as follows: According to the ship dynamic monitoring model and the channel situation deduction system, the abnormal behaviors of the ship obtained by the abnormality detection module are integrated, real-time hydrological information and radar / AIS data are combined, a channel three-dimensional dynamic map is constructed on a digital twin platform, the latest navigation signal is broadcast to the ship by using a shipborne ad hoc network, ship navigation guidance and route tracking are performed, dangerous areas are bypassed, or tracking navigation is performed according to historical navigation tracks, auxiliary navigation risk decision-making is performed, in the case of an emergency of the ship, emergency event determination is performed according to a federated learning and emergency response mechanism, encrypted transmission is performed through a heterogeneous network fusion system, and finally, the ship terminal executes emergency berthing, directly controls emergency equipment, and realizes rapid response.

7. The multimodal edge intelligence based air, land, sea, space, and cyberspace holographic perception and collaborative decision system, as claimed in claim 1, wherein: The network layer further includes a heterogeneous data network, and the heterogeneous data network includes: A near-shore transmission module: 5G network is used to real-time return high-definition monitoring videos of the port, and fine monitoring of the berthing and unberthing process of the ship by the VTS center is supported, and a LoRa network is connected to sensors in mountainous and remote sea areas to realize low-cost monitoring data return; A far-sea transmission module: Beidou short message is used to ensure transmission of emergency collision avoidance instructions when satellite communication is interrupted, and the minimum safety communication requirement of the ocean-going ship is ensured; A microwave relay ad hoc network: a ship-buoy dynamic network is constructed to perform multi-ship cooperative collision avoidance and local data sharing; A space-air transmission module: low-orbit satellite Internet of Things is used to realize wide-area coverage communication of the unmanned aerial vehicle, high-altitude platform and satellite, and large-range sea monitoring and emergency response are performed.

8. The multimodal edge intelligence based air, land, sea, space, and cyberspace holographic perception and collaborative decision system, as claimed in claim 1, wherein: The application layer includes: A multi-source data fusion architecture module: an "air-space-ground-sea" integrated perception network is constructed to eliminate data islands; Nanosecond-level synchronization is realized through Beidou satellites and atomic clocks to realize alignment of time and space references; A machine learning model is used to correct sea wave / electromagnetic interference deviation in real time to realize heterogeneous data calibration; Satellite images and radar echo superposition analysis are used to reduce ship positioning errors; Underwater acoustic signals and AIS data cross-validation are used to increase target recognition rate.

9. The multimodal edge intelligence based air, land, sea, space, and cyberspace holographic perception and collaborative decision system, as claimed in claim 1, wherein: The application layer further includes a data hierarchical model module, The data hierarchical model module includes: A data operation layer: raw data is collected in real time, and multi-source raw data of space-based, air-based, sea-based and land-based is converged; A data warehouse layer: a ship trajectory cubic module is included, a multi-dimensional model is constructed according to time, space and indexes, and is used for ship parallel analysis; A marine environment data cubic module: minute-level sampling is performed according to time, space and indexes to cover major global shipping lanes; Sensor failure prediction in the equipment health evaluation model library: a system equipment self-diagnosis model is established to monitor conditions of each sensor node, node conditions are analyzed based on signal quality, and a diagnosis report containing operation suggestions is generated by using historical fault cases and an expert rule base to perform pattern matching and root cause analysis on abnormal patterns, and is sent to a self-adjusting end to achieve effective predictive maintenance of the system, and external malicious attacks on the system are identified, gateway traffic is monitored, and corresponding protection measures are started to protect and isolate the system; A network node health index: communication link stability and edge computing node load rate are evaluated; A data application layer includes: Real-time traffic control application module: input ship trajectory cube and AIS historical data through ship abnormal behavior detection model, output yawing warning, overspeed warning and AIS silence detection probability, support real-time decision of collision warning system, dynamic channel capacity prediction combines VTS radar ship density and weather cube data, predicts channel capacity by time period, optimizes ship traffic flow organization; Environmental risk early warning application module: input SAR image target features and weather radar precipitation data through marine emergency response decision tree, output ship avoidance path suggestion and search and rescue priority ranking, support collision warning in extreme weather, analyze acoustic monitoring array signal features and historical submarine activity patterns through underwater target threat assessment, generate threat level heat map, avoid underwater obstacle collision; Network optimization application module: according to 5G base station load, satellite link state, output multi-path optimization scheme, ensure real-time transmission of collision warning instruction, edge computing resource scheduling model predicts port camera video traffic and buoy sensor data volume, allocates edge node computing power, guarantees data preprocessing efficiency.

Images