A holographic navigation scene graph system and construction method for ship intelligent navigation

Through the design of the holographic navigation scene diagram system, the problem of multi-dimensional dynamic information service requirements in intelligent ship navigation is solved, and high-precision navigation and navigation decision-making functions are realized. Through real-time collection and unified spatial and temporal reference methods, the information of intelligent navigation is met. Service requirements.

CN115017246BActive Publication Date: 2024-11-22WUHAN UNIV OF TECH

Patent Information

Application Number
CN202210597090.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-05-30
Publication Date
2024-11-22
Estimated Expiration
2042-05-30

AI Technical Summary

Technical Problem

The existing technology cannot effectively meet the multi-dimensional dynamic information service needs of ship intelligent navigation. The electronic channel map data is slow to update, and it cannot express the deep correlation of water transportation elements, making it difficult to meet the information service needs of intelligent navigation.

Method used

A holographic navigation scene diagram system was designed, including object label layer, information layer, element layer, object layer, scene layer and decision-making layer. Through the channel perception system, navigation scene data is collected in real time, space-time reference is unified, elements are extracted, and objects are established. and scenes, form a multi-dimensional dynamic expression scenario, and use a K8S-based microservice system to manage and visually present data.

Benefits of technology

It realizes adaptive multi-dimensional dynamic services for ship navigation tasks, provides high-precision navigation positioning, route planning and navigation decision-making functions, meets the information service needs of intelligent navigation, and obtains all-element information of navigation scenarios through ubiquitous perception, and realizes information Real-time and efficient management.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115017246B_ABST
    Figure CN115017246B_ABST
Patent Text Reader

Abstract

The present invention provides a holographic navigation scene diagram system and construction method for ship intelligent navigation. In view of the problems that the current electronic navigation chart data is updated slowly, cannot express the deep association of various water traffic elements, and is difficult to meet the information service needs of intelligent navigation, the system obtains the full element information of the navigation scene in real time or near real time through ubiquitous sensing means, obtains meteorological and hydrological information, traffic information, etc. related to navigation in geographic space, and associates the scene element information through semantic position, so as to realize the function of providing multi-dimensional dynamic services for ship navigation tasks. According to the application needs of the ship, the present invention integrates and associates the ubiquitous information of the navigation environment and water traffic objects related to the ship's behavior and activities with the object position as the link, and associates the ubiquitous information to the multi-dimensional dynamic scene for aggregation, analysis, transmission, expression and application, so as to meet the information service needs of intelligent navigation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of holographic navigation scenes, and in particular relates to a holographic navigation scene diagram system and a construction method for ship intelligent navigation. Background Art

[0002] Traditional cartography uses a multi-layer geographic data combination method to organize and store geographic space based on topological relationships or spatial entity data models, and uses scientific symbol systems and geographic languages ​​to express the spatial structure and relationship of various geographic elements. With the advent of the information and big data era, the modeling elements, information calculation and expression forms of traditional cartography are difficult to meet the expression and analysis needs of future geography. Various types of information are independent of each other to form "data islands", lacking a carrier that can efficiently utilize ubiquitous information, integrate, associate and manage multi-source heterogeneous spatiotemporal data, and express them in a reasonable way according to different scenario requirements. As a typical product of cartography, maps have become an indispensable means of travel in people's daily lives, and their main users are humans. On the other hand, machine maps for objects such as self-driving cars and service robots are also becoming a current research hotspot, developing towards high precision and intelligence.

[0003] In the field of navigation, electronic nautical charts or channel charts currently mainly serve pilots, providing rich geographical information such as water depth and obstacles for ship positioning and path planning. Humans can clearly understand all kinds of static and dynamic information required for navigation by relying on their own cognitive intelligence. Domestic and foreign scholars have conducted a lot of research on the application of electronic nautical charts. Tsou MC proposed a method for ship collision avoidance route planning based on the electronic chart display and information system (ECDIS), which can simultaneously avoid collision and obstacle avoidance of multiple ships. Different from the previous method of combining paper charts and tide tables to obtain water depth, Du Jiayun proposed a method combining FVCOM (Finite Volume Coast and Ocean Model) tidal numerical model and residual water level to achieve high-precision water depth solution services. Zhang Shanghong and others developed a digital waterway system platform based on three-dimensional virtual reality and network database technology, such as Figure 1As shown in the figure, the platform uses a two-dimensional hydrodynamic model based on shared memory mode (OpenMP) and a flow field visualization software based on textures, which improves the efficiency of water flow calculation and display accuracy. Pan Mingyang and others combined NASA's open source virtual earth World Wind and electronic navigation chart technology to carry out research on multi-dimensional digital channel models and Web multi-dimensional digital channel monitoring platforms with spatial information fusion, improving the real-time monitoring and management capabilities of ships, navigation marks, water levels and other elements.

[0004] For intelligent ship navigation, the role of holographic navigation scene images is mainly reflected in the following aspects:

[0005] (1) As the long-term memory of smart ships, it provides prior information for smart navigation of ships. Compared with shipborne sensors, the holographic navigation scene map is not restricted by weather conditions, obstacles, and operating distance, providing safety redundancy for smart ships. At the same time, the scene map can provide prior information for ship acceleration, deceleration, ballasting and other decisions.

[0006] (2) Gathering all information related to the ship's navigation mission and fully describing various elements in the navigation scene is an important way to achieve machine cognition of the navigation scene. The scene graph includes not only the spatial position and geometric information of traffic objects, but also semantic information such as the attributes of the objects themselves and the relationships between objects.

[0007] (3) Compared with traditional electronic nautical charts, holographic navigation scene diagrams can meet the requirements of information real-time, richness, accuracy and safety required for intelligent navigation decision-making. They are oriented to navigation tasks and provide users (humans or machines) with adaptive and customized personalized services. They are an important carrier of intelligent navigation in the ship-shore collaborative environment. Summary of the invention

[0008] The technical problem to be solved by the present invention is to provide a holographic navigation scene graph system and construction method for ship intelligent navigation, which is used to adaptively provide multi-dimensional dynamic services for ship navigation tasks.

[0009] The technical solution adopted by the present invention to solve the above technical problems is:

[0010] A holographic navigation scene diagram system for intelligent navigation methods, wherein the system model includes, from the bottom to the top, an object layer, an information layer, an element layer, an object layer, a scene layer and a decision layer; the system hardware includes a data acquisition module, a data processing module, a data storage module and a data service module; the data acquisition module includes a channel perception system for online real-time monitoring of navigation scenes through ubiquitous perception, collecting elements of navigation scenes, and constructing information constituting the object layer, the information layer and the element layer; the data processing module is used to process the received data by unifying spatiotemporal references, classifying and extracting elements, establishing objects, constituting object behaviors and scenes, forming scene decision knowledge, and multi-dimensionally expressing scenes dynamically. data; the data processing module includes a timing module, which is used to unify the time base of each device; the storage module includes a database system, which is used to quickly process and publish big data of navigation scenes; the data service module adopts a microservice system based on a holographic navigation scene graph of K8S, which is used to manage the basic data of the holographic navigation scene graph, visualize the navigation scene data, retrieve the waterway element information, and dynamically push the inland waterway traffic data; the data service module includes a waterway integrated management system, a navigation safety early warning system, and a host network and information security system; the waterway integrated management system is used to integrate various waterway operation status information; the navigation safety early warning system is used to ensure the safety of ship navigation operations.

[0011] According to the above plan, the channel perception system includes oblique photography, high-definition video or infrared cameras and their ancillary equipment, lidar, millimeter-wave radar, water level observation instrument, flow direction observation instrument, flow velocity observation instrument, underwater multi-beam depth sounding system, meteorological observation instrument, visibility meter and ship AIS equipment.

[0012] According to the above scheme, the elements of the navigation scene include time, space, objects, things, phenomena and events, which are used to reflect the spatial differentiation, evolution process and interaction relationship between water traffic objects and water traffic events in the navigation scene; the elements are divided into static elements and dynamic elements;

[0013] Static elements include channel elements, shore-based environmental elements and navigation facilities elements; underwater three-dimensional terrain and water depth data in channel elements are obtained through underwater multi-beam bathymetry system; shore-based environmental elements including shoreline, bridges, and shore-based significant objects are collected through cameras; navigation facilities elements including berths, anchorage distribution and their scale are obtained through electronic channel charts and database integration;

[0014] Dynamic elements include traffic elements, meteorological elements and hydrological elements; traffic elements include regional traffic density, ship position, heading and speed data, which are collected through AIS or Qianxun, camera, laser radar and millimeter wave radar; ships with AIS use AIS to obtain their status information; ships without AIS and other dynamic objects are detected by laser radar and millimeter wave radar on shore-based sensing equipment, and the generated three-dimensional point cloud data is processed and superimposed on the electronic river map to realize the update of dynamic objects; meteorological elements include temperature, humidity, wind direction, wind speed, air pressure, precipitation and visibility, which are collected through meteorological observation instruments and visibility instruments; hydrological elements include water level, flow direction and flow velocity of the channel, which are collected through water level observation instruments, flow direction observation instruments and flow velocity observation instruments respectively;

[0015] The object layer includes the characteristic objects and spatial objects of the elements, including the points, lines and surfaces of the objects; the information layer includes the static attribute data and dynamic attribute data of the elements; the element layer includes the type, attributes and distribution of the elements in the current scene.

[0016] A method for constructing a holographic navigation scene graph system comprises the following steps:

[0017] S0: Build a holographic navigation scene graph system for intelligent navigation. The system model includes the object layer, information layer, element layer, object layer, scene layer and decision layer from the bottom to the top. The system hardware includes data acquisition module, data processing module, data storage module and data service module. The data acquisition module includes a waterway perception system; the data processing module includes a timing module; the data storage module includes a database system; the data service module adopts a microservice system based on the holographic navigation scene graph of K8S; the data service module includes a waterway integrated management system, a navigation safety warning system, and a host network and information security system.

[0018] S1: The data acquisition module obtains online real-time monitoring data of the navigation scene through the infrastructure perception network, information network, and service network, and establishes the organizational structure of the system in the form of object layer, information layer, and element layer;

[0019] S2: The data processing module processes the received data by unifying spatiotemporal references, classifying and extracting elements, establishing objects, constituting object behaviors and scenarios, forming scenario decision knowledge, and expressing scenarios in a multi-dimensional and dynamic manner;

[0020] S3: The data storage module uses ClickHouse's column storage and data compression to store data, uses relational models to describe data and express the parallel relationship between various attributes of ship trajectory data, sets multiple primary keys and multi-level indexes to optimize the speed of regional queries; uses the H3 hexagonal hierarchical grid indexing method to propose a ship trajectory data storage method and design a data storage mode, and calculates the H3 grid code value at any longitude and latitude to reduce the amount of data scanning;

[0021] S4: The data service module is based on the holographic navigation scene map, manages the basic data of the holographic navigation scene map, visualizes the navigation scene data, retrieves the waterway element information, and dynamically pushes the inland waterway traffic data.

[0022] Furthermore, in step S2, the specific steps of unifying the spatiotemporal reference are:

[0023] S201: The timing module sends the same time reference to the sensors that support external trigger sampling to unify the time coordinate system; for sensors that do not support external trigger sampling, when the operating frequencies are inconsistent, the asynchronous sampling results are interpolated or extrapolated to achieve time alignment at the software level;

[0024] S202: The position and posture of the object in space and time are expressed in analytical form as spatial coordinates; in the three-dimensional spatial coordinate system, the position of the object is expressed by three orthogonal axes X, Y, and Z, and the posture of the object is expressed by the rotation angles around these three orthogonal axes, including the roll angle roll, the pitch angle pitch, and the yaw angle yaw; the spatial coordinate system includes the ship's onboard coordinate system, the local coordinate system of the shore-based sensor, and the world coordinate system based on the geodetic coordinates; static elements are expressed by the geodetic coordinate system; dynamic elements are expressed by different coordinate systems according to different application scenarios.

[0025] Furthermore, in step S2, the specific steps of classifying and extracting elements are as follows:

[0026] S203: According to the characteristics of the collected scene data elements and the water traffic scene, different typical scene types are divided based on the location; the typical location scene types include open waters, waterways, bridge areas, anchorages, berths, and lock waters;

[0027] S204: further dividing the scene graph elements into specific ship navigation tasks or functional scenarios: ship-shore contact warning, bridge contact warning, obstacle warning, grounding warning, ship collision warning, channel centerline prompt, ship towing prompt, assisted berthing and unberthing, assisted lock passing, perception enhancement, lane-level route planning and speed optimization;

[0028] S205: setting a sliding window that moves with the ship's navigation, and extracting only feature objects within the sliding window when the ship is sailing; the window is a rectangle that includes a certain water area around the ship, and the size of the rectangle is set according to the length and width of the ship's area at a normal speed, and is dynamically adjusted according to the size of the ship's speed. The greater the ship's speed, the larger the area of ​​the sliding window;

[0029] Ships pay attention to different scene elements when performing different navigation tasks; when planning routes, they pay attention to key turning points, water depth and bridge clearance; when sailing along the channel, they pay attention to the dynamics of ships far ahead; when encountering other ships, they pay attention to the dynamics of ships that pose a risk of collision; when berthing or leaving the berth, they pay attention to the dynamics of the ship, berth position and water flow element information;

[0030] According to the needs of the navigation task, the data gathered in the sliding window is reorganized to extract the key elements related to the navigation task, and the automatic matching of the task and the elements of interest is realized;

[0031] S206: Continuously update the scene graph data according to the changes in the ship's navigation position, and inform the ship's automatic driving system of the current position and nearby scene information.

[0032] Furthermore, in step S2, the specific steps of establishing the object are:

[0033] S207: Fusing the extracted information; fusing the dynamic information of the ship including the position, heading and speed for high-precision positioning of the ship; fusing the data including GNSS, point cloud and image for identifying the elements in the scene including shore base, bridge, water surface, ship, navigation mark and obstacle;

[0034] S208: combining elements and attributes to form an object; encapsulating elements and attributes or combining elements with elements according to the classes in the elements to form an object in the water transportation scene; extracting various attributes of the same object from different layers of the elements, binding the elements and attributes, and forming a semantic description of the object.

[0035] Furthermore, in step S2, the specific steps of constructing the object behavior and scene are:

[0036] S209: dividing the relationships between objects in the water transportation scene into spatial relationships, temporal relationships and semantic relationships, and constructing object behaviors;

[0037] In view of the dynamic behavior of continuous action, the dynamic model is used to express and describe the dynamic changes of objects in time and space;

[0038] For the dynamic behavior of discrete actions, a set of rules is used to express the control and execution process of the behavior;

[0039] S210: Construct a scene; combine objects and behaviors to form a water traffic scene, and use a graph model to organize and manage the reconstructed water traffic scene data; the graph model uses nodes, relationships, and labels to represent objects and various relationships between objects, and represents them in the form of a graph structure; the scene objects, entity objects, and events in the water traffic scene are uniformly mapped to nodes, and all nodes integrate basic attributes including spatial reference, spatial range, time information, attribute set, and behavior information; the entity object node also includes the data set attributes of the traffic data held by the entity, and the data set itself is also stored as a data node; the event object node also records the participating entities, the evolution process of the participating entities, and the description of the role played in the event.

[0040] Furthermore, in step S2, the specific steps of forming scene decision knowledge are:

[0041] S211: For different combinations of objects and behaviors, role identification is performed on ship objects with specified behaviors and attributes in each water traffic rule scenario, and based on the ontology's reasoning and query mechanism, semantic query and reasoning are performed on the operations that each ship should perform, forming a rule knowledge graph with specific behaviors and attributes in a specific scenario.

[0042] Furthermore, in step S2, the specific steps of multi-dimensional dynamic expression scene are:

[0043] S212: Establish a user model based on the attributes, behaviors and relationships between different water transportation objects and other objects, customize visual scene graph templates that meet different needs, realize adaptive user scene graph expression in a limited scene graph space, and efficiently associate and express information that meets application scenario needs based on location;

[0044] S213: Extract relevant information about navigation-related objects and their attributes, and automatically perform calculations and visual expressions at different granularities.

[0045] The beneficial effects of the present invention are:

[0046] 1. The present invention provides a holographic navigation scene diagram system and construction method for intelligent navigation of ships. In order to solve the problems that the current electronic navigation chart data is slow to update, cannot express the deep association of various water traffic elements, and is difficult to meet the information service needs of intelligent navigation, the system obtains the full-element information of the navigation scene in real time or near real time through ubiquitous sensing means, obtains the meteorological and hydrological information, traffic information, etc. related to navigation in the geographic space, and associates the scene element information through semantic position, thereby realizing the function of providing multi-dimensional dynamic services for ship navigation tasks adaptively.

[0047] 2. According to the application requirements of ships, the present invention uses location as a link to integrate and associate ubiquitous information of navigation environment and water traffic objects related to ship behavior and activities, and associates the ubiquitous information into multi-dimensional dynamic scenes for aggregation, analysis, transmission, expression and application.

[0048] 3. The present invention provides object-oriented intelligent services such as high-precision navigation positioning, route planning, and navigation decision-making, meeting the information service needs of intelligent navigation. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a diagram of basic elements of an embodiment of the present invention.

[0050] Figure 2 It is a characteristic diagram of an embodiment of the present invention.

[0051] Figure 3 It is an architectural diagram of an embodiment of the present invention.

[0052] Figure 4 It is a principle block diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0053] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0054] See also Figure 4 , an embodiment of the present invention includes a data acquisition module, a data processing module, a data storage module and a data service module.

[0055] 1. Data Collection

[0056] The data acquisition module is used to obtain online real-time monitoring data with the help of infrastructure perception network, information network and service network, so as to improve the timeliness and authenticity of the ubiquitous information of the inland holographic navigation scene. The scene graph system includes: a waterway perception system including various sensors such as CCTV and radar, a waterway integrated management system integrating various waterway operation status information, a navigation safety warning system to ensure the safety of ship navigation operations, as well as a database system, a host network and information security system. The required sensing equipment and means include: oblique photography, high-definition video / infrared cameras and their ancillary equipment, laser radar, small phased array radar, water level observation instrument, flow velocity observation instrument, meteorological observation instrument, visibility meter and ship AIS equipment.

[0057] The data collected by ubiquitous sensing means are divided into static scene data and dynamic data. Static navigation scene data and its collection methods include: waterway elements, shore-based environmental elements and navigation infrastructure elements.

[0058] 1) The underwater three-dimensional terrain and water depth data in the channel elements are obtained through the underwater multi-beam bathymetry system;

[0059] 2) Real-life data of shore-based environmental elements such as shorelines, bridges, and significant landmarks on the shore are collected by cameras;

[0060] 3) Elements of navigation facilities such as berths, anchorage distribution and their scales are obtained through electronic navigation charts and database integration.

[0061] The drone is equipped with multiple visual sensors, which simultaneously collect images from five different angles, including one vertical and four inclined angles, to obtain the conditions of the surface features in the area near the waterway. Then, a real three-dimensional surface environment model is generated through positioning, fusion, and modeling. The underwater terrain is integrated with the surface environment model to achieve three-dimensional reconstruction of the static environment above and below the water.

[0062] Dynamic navigation scene data and its collection methods mainly include:

[0063] 1) Traffic elements: regional traffic density, ship position, heading, and speed data. Collected by AIS / Qianxun, camera, laser radar, and millimeter-wave radar. For ships with AIS, AIS is used to obtain their status information. For ships without AIS and other dynamic objects, the laser radar and millimeter-wave radar on the shore-based sensing equipment detect dynamic objects in the area, and the generated three-dimensional point cloud data is processed and superimposed on the electronic river map to update the dynamic objects.

[0064] 2) Meteorological elements: temperature, humidity, wind direction, wind speed, air pressure, precipitation, visibility, etc. Collected by six-element automatic weather stations and visibility meters.

[0065] 3) Hydrological elements: water level, flow direction and flow velocity of the waterway. Collected by water level, flow direction and flow velocity observation instruments.

[0066] After data collection is completed, Figure 3 The architecture is organized in the form of layers of object layer, information layer and element layer. The object layer refers to the characteristic objects and spatial objects of various elements, including points, lines and surfaces of objects. The information layer refers to the static and dynamic attribute data of various elements. The element layer refers to the types, attributes and distribution of various elements in the current scene.

[0067] Figure 3 The information at the object level and above is processed in the data processing module.

[0068] 2. Data processing

[0069] (1) Unified space-time reference

[0070] The time coordinate system is a one-dimensional time coordinate system. There are multiple sensors in the scene graph system, which generally have their own independent clocks. In order to establish a unified time coordinate system so that each device has the same time reference, a high-precision timing system is equipped in hardware. For sensors that do not support external triggering for sampling, when their operating frequencies are inconsistent, the asynchronous sampling results are interpolated or extrapolated to achieve time alignment at the software level.

[0071] Spatial coordinates are used to express the position and posture of an object in space and time in an analytical form (coordinates). In general, a three-dimensional spatial coordinate system uses three orthogonal axes X, Y, and Z to represent the position of an object, and uses the rotation angles around these three orthogonal axes (roll angle, pitch angle, and yaw angle) to represent the posture of the object. The spatial coordinate systems in the scene graph system include the ship's onboard coordinate system, the local coordinate system of shore-based sensors (such as the coordinate system of shore-based radars), and the world coordinate system based on geodetic coordinates. In practical applications, in order to obtain the spatial orientation and distance relationship between ships and other objects, it is necessary to unify the spatial coordinate system, associate different coordinate systems, and establish conversion relationships between them. Static elements such as waterway elements, shore-based environmental elements, and navigation facility elements are represented by the geodetic coordinate system; dynamic traffic elements can be expressed in different coordinate systems according to different application scenarios. For example, when ship A is detected by a shore-based radar, the position of ship A is calibrated using the local coordinate system of the shore-based radar; when ship B needs the spatial relationship between itself and ship A, it is necessary to project ship A into the ship-borne coordinate system of ship B for expression; when it is necessary to obtain the spatial relationship between ship A, ship B and other nearby traffic objects, it is necessary to convert their positions and postures into a unified geodetic coordinate system through a transformation relationship for expression.

[0072] (2) Feature classification and extraction

[0073] 1) Based on the characteristics of the six types of scene data elements and water traffic scenes collected, the scene graph elements in different scenes are divided based on location. The scenes in different locations mainly include: open waters, waterways, bridge areas, anchorages, berths, and lock waters.

[0074] 2) Since there are still many types of traffic scenes in the same location, it is difficult to quickly extract the location scene, so it is necessary to further divide the specific ship navigation tasks or functional scenes. The ship intelligent navigation functional scenes are divided into: ship-shore contact warning, bridge contact warning, obstacle warning, grounding warning, ship collision warning, channel centerline prompt, ship towing prompt, assisted berthing, assisted lock passing, perception enhancement, lane-level route planning and speed optimization.

[0075] 3) Set a sliding window that moves with the ship's navigation. The window is a rectangle that includes a certain water area around the ship. At normal speed, it can be set according to the length and width of the ship's area, and dynamically adjusted according to the size of the ship's speed. When the ship's speed is high, the area of ​​the sliding window is appropriately increased. When the ship is sailing, only the element objects within the sliding window are extracted. In addition, the scene elements that ships pay attention to when performing different navigation tasks are different. According to the needs of the navigation task, the data gathered in the sliding window is reorganized to extract the key elements related to the navigation task. When planning the route, only the key turning points, water depth, and bridge clearance need to be paid attention to; when sailing along the waterway, the ship pays attention to a wider field of vision, mainly focusing on the dynamics of the ship in the distance; when meeting with other ships, it is necessary to pay attention to the dynamics of the ship that poses a collision risk; when berthing and leaving, the main focus is on the dynamics of the ship, berth position, water flow and other element information. Therefore, for different navigation tasks, it is necessary to extract key information from the gathered data, filter redundant information, and realize the automatic matching of tasks and elements of concern.

[0076] By setting a sliding window around the ship, the scene graph data is continuously updated according to the changes in the ship's navigation position, thereby informing the autonomous driving system of the current position of the ship and the scene information near that position.

[0077] (3) Object creation

[0078] Information fusion after extraction. The element data in the sliding window is multi-source and heterogeneous, and different levels of data fusion are required to ensure the reliability of the information. It is mainly aimed at 1) the fusion of dynamic information such as ship position, heading, speed, etc. to achieve high-precision positioning of ships; 2) the fusion of GNSS, point cloud, image and other data to identify shore bases, bridges, water surfaces, ships, navigation marks, obstacles and other elements in the scene.

[0079] The combination of features and attributes forms an object. According to the class in the feature, the features and attributes are encapsulated or combined with features to form an object in the water transportation scene. That is, various attributes of the same object are extracted from different layers of the feature, and the features and attributes are bound to form a semantic description of the object. For example, the boundary line, water depth and other attributes of the waterway are represented on different layers, while in the object layer, the waterway landmark itself, the boundary line of the waterway, the water depth and other attributes together constitute the waterway object.

[0080] (4) Object Behavior and Scene Composition

[0081] The relationships between objects in water traffic scenes are divided into spatial relationships, temporal relationships, and semantic relationships. Behaviors mainly reflect changes in object attributes and relationships. For dynamic behaviors with continuous effects, dynamic models are used to express and describe the dynamic changes of spatiotemporal objects. For example, for meteorological and hydrological entities, corresponding field patterns can be constructed to characterize changes. For dynamic behaviors with discrete effects, rule sets are used to express the control and execution process of behaviors. For example, for navigation facility entities, state changes can be identified through rule detection. For dynamic behaviors with unknown action processes but a large amount of historical sample data, statistical analysis and data mining methods are introduced to mine time series patterns of historical data, identify specific behaviors and behavior sequences, and establish the relationship between behaviors and intrinsic attributes. For example, for ship entities, historical trajectory patterns can be mined to identify specific navigation routes and navigation characteristics.

[0082] The composition of the scene. The combination of objects and behaviors constitutes the underwater traffic scene, and the graph model is used to organize and manage the reconstructed water traffic scene data. The graph model uses nodes, relationships, and labels to represent objects and various associations between objects, and is represented in the form of a graph structure. The scene objects, entity objects, and events in the water traffic scene are uniformly mapped to nodes. All nodes integrate basic attributes such as spatial reference, spatial range, time information, attribute set, and behavior information; the entity object node also contains the data set attributes of the traffic data held by the entity, and the data set itself is also stored as a data node; the event object node also records the participating entities, the evolution process of the participating entities, and the description of the role played in the event.

[0083] (5) Formation of scenario decision-making knowledge

[0084] According to different combinations of objects and behaviors, the role of the ship object with specified behaviors and attributes in each water traffic rule scene is identified, and based on the ontology's reasoning and query mechanism, the operations that each ship should perform are semantically queried and reasoned, forming a rule knowledge graph with specific behaviors and attributes in a specific scene. After inputting information such as shore-based environment, channel conditions, hydrological and meteorological conditions, and ship dynamics, the intelligent ship realizes the scene type recognition and behavior decision-making functions in the rules based on the query and reasoning functions of the knowledge graph.

[0085] (6) Multi-dimensional dynamic expression of scenes

[0086] 1) Multi-perspective switching: In a limited scene graph space, information that meets the needs of the application scenario is efficiently associated and expressed based on location. For ship navigation, it provides a navigation perspective to crew members and remote control personnel; for waterway supervision, it provides a "God's perspective" to supervisors. For different water transportation objects, according to their attributes, behaviors, and relationships with other objects, a user model is established, and scene graph visualization templates that meet different needs are customized to achieve adaptive user scene graph expression.

[0087] 2) Adaptation. The users of the holographic navigation scene graph include autonomous ships. The navigation scenes of ships are complex and diverse. In order to meet the navigation needs of ships with different degrees of automation in different scenes, the scene graph intelligently extracts relevant information such as navigation-related objects and their attributes, and then automatically performs calculations and visualizations of different granularities. For example, in the scene where a ship is sailing in open waters under good weather conditions, the ship mainly focuses on objects such as ships in the distance, and only needs to perform coarse-grained calculations, and the chart range and radar range will be automatically adjusted to a larger level accordingly.

[0088] 3. Data Storage

[0089] The data storage module is mainly responsible for the rapid processing and publishing of navigation scene big data. In view of the high frequency, massiveness and multidimensionality of ship dynamic data, and the query needs of ship trajectory data, ClickHouse's unique column storage and data compression features are used to effectively reduce the data scanning range and the size of data transmission. The relational model is used to describe the data to express the parallel relationship between the various attributes of the ship trajectory data, and multiple primary keys and multi-level indexes are set to optimize the speed of regional queries. The H3 hexagonal hierarchical grid indexing method is used to propose a ship trajectory data storage method and design a data storage mode to calculate the H3 grid code value at any longitude and latitude, thereby reducing the amount of data scanning.

[0090] 4. Data Services

[0091] The data service module is a microservice system of the holographic navigation scene graph based on K8S, which realizes functions such as basic data management of the holographic navigation scene graph, visualization of navigation scene data, waterway element information retrieval service, and dynamic push of inland waterway traffic data.

[0092] The above embodiments are only used to illustrate the design ideas and features of the present invention, and their purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly. The protection scope of the present invention is not limited to the above embodiments. Therefore, any equivalent changes or modifications made based on the principles and design ideas disclosed by the present invention are within the protection scope of the present invention.

Claims

1. A holographic navigation scene diagram system for intelligent navigation of ships, characterized by: The system model includes object layer, information layer, element layer, object layer, scene layer and decision layer from bottom to top; the system hardware includes data acquisition module, data processing module, data storage module and data service module; The data collection module includes a waterway perception system, which is used to monitor the navigation scene in real time online through ubiquitous perception, collect the elements of the navigation scene, and construct information that constitutes the object layer, information layer, and element layer; The data processing module is used to process the received data by unifying the time and space reference, classifying and extracting elements, establishing objects, constituting object behaviors and scenes, forming scene decision knowledge, and multi-dimensional dynamic expression scenes; the data processing module includes a timing module, which is used to unify the time base of each device; The data storage module includes a database system for rapid processing and publishing of big data of navigation scenarios; The data service module uses a microservice system based on the holographic navigation scene graph of K8S, which is used to manage the basic data of the holographic navigation scene graph, visualize the navigation scene data, retrieve the waterway element information, and dynamically push the inland waterway traffic data; The data service module includes the waterway integrated management system, navigation safety warning system, host network and information security system; the waterway integrated management system is used to integrate various waterway operation status information; the navigation safety warning system is used to ensure the safety of ship navigation operations; The channel perception system includes oblique photography, high-definition video or infrared cameras and their ancillary equipment, laser radar, millimeter-wave radar, water level observation instrument, flow direction observation instrument, flow velocity observation instrument, underwater multi-beam bathymetric system, meteorological observation instrument, visibility meter and ship AIS equipment; The elements of a navigation scene include time, space, objects, things, phenomena and events, which are used to reflect the spatial differentiation, evolution process and interaction relationship between water traffic objects and water traffic events in the navigation scene; the elements are divided into static elements and dynamic elements; Static elements include channel elements, shore-based environmental elements and navigation facilities elements; underwater three-dimensional terrain and water depth data in channel elements are obtained through underwater multi-beam bathymetry system; shore-based environmental elements including shoreline, bridges, and shore-based significant objects are collected through cameras; navigation facilities elements including berths, anchorage distribution and their scale are obtained through electronic channel charts and database integration; Dynamic elements include traffic elements, meteorological elements and hydrological elements; traffic elements include regional traffic density, ship position, heading and speed data, which are collected through AIS or Qianxun, camera, laser radar and millimeter wave radar; ships with AIS use AIS to obtain their status information; ships without AIS and other dynamic objects are detected by laser radar and millimeter wave radar on shore-based sensing equipment, and the generated three-dimensional point cloud data is processed and superimposed on the electronic river map to realize the update of dynamic objects; meteorological elements include temperature, humidity, wind direction, wind speed, air pressure, precipitation and visibility, which are collected through meteorological observation instruments and visibility instruments; hydrological elements include water level, flow direction and flow velocity of the channel, which are collected through water level observation instruments, flow direction observation instruments and flow velocity observation instruments respectively; The object layer includes the characteristic objects and spatial objects of the elements, including the points, lines and surfaces of the objects; the information layer includes the static attribute data and dynamic attribute data of the elements; the element layer includes the type, attributes and distribution of the elements in the current scene.

2. A method for constructing a holographic navigation scene graph system for ship intelligent navigation according to claim 1, characterized in that: The following steps are involved: S0: Build a holographic navigation scene graph system for intelligent navigation. The system model includes the object layer, information layer, element layer, object layer, scene layer and decision layer from the bottom to the top. The system hardware includes data acquisition module, data processing module, data storage module and data service module. The data acquisition module includes a waterway perception system; the data processing module includes a timing module; the data storage module includes a database system; the data service module adopts a microservice system based on the holographic navigation scene graph of K8S; the data service module includes a waterway integrated management system, a navigation safety warning system, and a host network and information security system. S1: The data acquisition module obtains online real-time monitoring data of the navigation scene through the infrastructure perception network, information network, and service network, and establishes the organizational structure of the system in the form of object layer, information layer, and element layer; S2: The data processing module processes the received data by unifying spatiotemporal references, classifying and extracting elements, establishing objects, constituting object behaviors and scenarios, forming scenario decision knowledge, and expressing scenarios in a multi-dimensional and dynamic manner; S3: The data storage module uses ClickHouse's column storage and data compression to store data, uses a relational model to describe data and express the parallel relationship between various attributes of ship trajectory data, and sets multiple primary keys and multi-level indexes to optimize the speed of regional queries; Using the H3 hexagonal hierarchical grid indexing method, a ship trajectory data storage method is proposed and a data storage mode is designed to calculate the H3 grid code value at any longitude and latitude to reduce the amount of data scanning. S4: The data service module is based on the holographic navigation scene map, manages the basic data of the holographic navigation scene map, visualizes the navigation scene data, retrieves the waterway element information, and dynamically pushes the inland waterway traffic data.

3. The construction method according to claim 2, characterized in that: In step S2, the specific steps of unifying the spatiotemporal reference are: S201: The timing module sends the same time reference to the sensors that support external trigger sampling to unify the time coordinate system; for sensors that do not support external trigger sampling, when the operating frequencies are inconsistent, the asynchronous sampling results are interpolated or extrapolated to achieve time alignment at the software level; S202: The position and posture of the object in space and time are expressed in analytical form as spatial coordinates; in the three-dimensional spatial coordinate system, the position of the object is expressed by three orthogonal axes X, Y, and Z, and the posture of the object is expressed by the rotation angles around these three orthogonal axes, including the roll angle roll, the pitch angle pitch, and the yaw angle yaw; the spatial coordinate system includes the ship's onboard coordinate system, the local coordinate system of the shore-based sensor, and the world coordinate system based on the geodetic coordinates; static elements are expressed by the geodetic coordinate system; dynamic elements are expressed by different coordinate systems according to different application scenarios.

4. The construction method according to claim 2, characterized in that: In step S2, the specific steps of classifying and extracting elements are: S203: According to the characteristics of the collected scene data elements and the water traffic scene, different typical scene types are divided based on the location; the typical location scene types include open waters, waterways, bridge areas, anchorages, berths, and lock waters; S204: further dividing the scene graph elements into specific ship navigation tasks or functional scenarios: ship-shore contact warning, bridge contact warning, obstacle warning, grounding warning, ship collision warning, channel centerline prompt, ship towing prompt, assisted berthing and unberthing, assisted lock passing, perception enhancement, lane-level route planning and speed optimization; S205: setting a sliding window that moves with the ship's navigation, and extracting only feature objects within the sliding window when the ship is sailing; the window is a rectangle that includes a certain water area around the ship, and the size of the rectangle is set according to the length and width of the ship's area at a normal speed, and is dynamically adjusted according to the size of the ship's speed. The greater the ship's speed, the larger the area of ​​the sliding window; Ships pay attention to different scene elements when performing different navigation tasks; when planning routes, they pay attention to key turning points, water depth and bridge clearance; when sailing along the channel, they pay attention to the dynamics of ships far ahead; when encountering other ships, they pay attention to the dynamics of ships that pose a risk of collision; when berthing or leaving the berth, they pay attention to the dynamics of the ship, berth position and water flow element information; According to the needs of the navigation task, the data gathered in the sliding window is reorganized to extract the key elements related to the navigation task, and the automatic matching of the task and the elements of interest is realized; S206: Continuously update the scene graph data according to the changes in the ship's navigation position, and inform the ship's automatic driving system of the current position and nearby scene information.

5. The construction method according to claim 2, characterized in that: In step S2, the specific steps of establishing the object are: S207: Fusing the extracted information; fusing the dynamic information of the ship including the position, heading and speed for high-precision positioning of the ship; fusing the data including GNSS, point cloud and image for identifying the elements in the scene including shore base, bridge, water surface, ship, navigation mark and obstacle; S208: combining elements and attributes to form an object; encapsulating elements and attributes or combining elements with elements according to the classes in the elements to form an object in the water transportation scene; extracting various attributes of the same object from different layers of the elements, binding the elements and attributes, and forming a semantic description of the object.

6. The construction method according to claim 2, characterized in that: In step S2, the specific steps of constructing object behaviors and scenes are: S209: dividing the relationships between objects in the water transportation scene into spatial relationships, temporal relationships and semantic relationships, and constructing object behaviors; In view of the dynamic behavior of continuous action, the dynamic model is used to express and describe the dynamic changes of objects in time and space; For the dynamic behavior of discrete actions, a set of rules is used to express the control and execution process of the behavior; S210: constructing a scene; combining objects and behaviors to form a water traffic scene, and using a graph model to organize and manage the reconstructed water traffic scene data; the graph model uses nodes, relationships, and labels to represent objects and various associations between objects, and is represented in the form of a graph structure; The scene objects, entity objects and events in the water traffic scene are uniformly mapped into nodes. All nodes integrate basic attributes including spatial reference, spatial range, time information, attribute set and behavior information. The entity object node also contains the data set attributes of the traffic data held by the entity, and the data set itself is also stored as a data node. The event object node also records the participating entities, the evolution of the participating entities, and a description of the role they play in the event.

7. The construction method according to claim 2, characterized in that: In step S2, the specific steps of forming scene decision knowledge are: S211: For different combinations of objects and behaviors, role identification is performed on ship objects with specified behaviors and attributes in each water traffic rule scenario, and based on the ontology's reasoning and query mechanism, semantic query and reasoning are performed on the operations that each ship should perform, forming a rule knowledge graph with specific behaviors and attributes in a specific scenario.

8. The construction method according to claim 2, characterized in that: In step S2, the specific steps of multi-dimensional dynamic expression scene are: S212: Establish a user model based on the attributes, behaviors and relationships between different water transportation objects and other objects, customize visual scene graph templates that meet different needs, realize adaptive user scene graph expression in a limited scene graph space, and efficiently associate and express information that meets application scenario needs based on location; S213: Extract relevant information about navigation-related objects and their attributes, and automatically perform calculations and visual expressions at different granularities.

Citation Information

Patent Citations

  • Hybrid live-action three-dimensional navigation channel scene construction method

    CN113538679A

  • Ship collision avoidance rule scene knowledge formalized expression method

    CN114443979A

  • Waterborne traffic ontology modeling method and model for holographic navigation scene graph

    CN114969928A

  • Holographic navigation scene graph construction method and device based on ontology modeling

    CN114972666A

Cited By

  • Quantitative model-based shipping risk assessment method and system

    CN120125044A

  • A shipping risk assessment method and system based on a quantization model

    CN120125044B