A maritime space-time knowledge graph visualization method and system

Abstract

This application belongs to the field of visualization, specifically disclosing a method and system for visualizing a maritime spatiotemporal knowledge graph. The method includes: dividing the projection plane of Earth's latitude and longitude space into multiple spatial grids and encoding them; dividing it into equal-length time segments in the time dimension and encoding them; merging the spatial and temporal codes to obtain a spatiotemporal grid code; using the spatiotemporal grid code as a hash key to construct a hash index table associating the spatiotemporal grid code with the unique identifier of the entity; rendering the entity as a three-dimensional sphere located within the corresponding three-dimensional cube according to the hash index table, and drawing three-dimensional curves between the three-dimensional spheres of the associated entities according to the triple data; dynamically switching the current spatiotemporal grid and updating the visualization content in response to user interaction commands. Through this application, the spatiotemporal relationships and triple relationships between entities in the graph can be clearly expressed, improving the comprehensiveness of knowledge graph visualization.

Description

Technical Field

[0001] This application belongs to the field of visualization technology, and more specifically, relates to a method and system for visualizing maritime spatiotemporal knowledge graphs. Background Technology

[0002] Spatiotemporal knowledge graphs are a new form of knowledge graph that has emerged in recent years, building upon traditional knowledge graphs and focusing on the mining and reasoning applications of spatiotemporal relationships between entities. Unlike traditional triple-based knowledge graphs, spatiotemporal knowledge graphs emphasize spatial location and temporal characteristics among entity attributes, and highlight the discovery and mining of spatial, temporal, and spatiotemporal joint relationships between entities. This need is particularly urgent in maritime spatiotemporal knowledge graphs designed for maritime applications.

[0003] Therefore, highlighting the spatiotemporal relationships between entities while showcasing traditional relationships has become a key challenge in the visualization technology of maritime spatiotemporal knowledge graphs. Existing spatiotemporal knowledge graph visualization technologies mainly fall into two categories. One category uses traditional graph nodes and edges, abstracting spatiotemporal attributes into graph nodes and spatiotemporal relationships between entities into edges, using existing graph visualization models to display these relationships. This approach can display relationships between traditional graph nodes and spatiotemporal relationships, but it cannot clearly express the spatiotemporal connections between different spatiotemporal relationships. The other category is based on a GIS system, projecting entity nodes onto the GIS system based on the spatial location of the entities. This approach can clearly show the spatial location relationships between entities but cannot express temporal connections, and it is also difficult to display traditional graph relationships.

[0004] Therefore, there is an urgent need for a visualization method that can clearly express the spatiotemporal relationships between graph entities as well as the traditional graph entity triplet relationships. Summary of the Invention

[0005] In view of the shortcomings of existing technologies, the purpose of this application is to achieve a maritime spatiotemporal knowledge graph that retains the traditional knowledge node relationships while clearly displaying the relationships between entities in the spatial, temporal, and spatiotemporal joint dimensions.

[0006] To achieve the above objectives, firstly, this application provides a method for visualizing maritime spatiotemporal knowledge graphs, including: The projection plane of Earth's latitude and longitude space is divided into multiple spatial grids and encoded to obtain a multi-scale pyramid structure spatial code. It is also divided into equal-length time segments in the time dimension and encoded to obtain a time code. The spatial code and the time code are merged to obtain a spatiotemporal grid code. Using the spatiotemporal grid code as a hash key, a hash index table is constructed that associates the spatiotemporal grid code with the entity's unique identifier, and triple data representing the relationship between entities is obtained and stored. Based on a three-dimensional digital earth, a three-dimensional cube corresponding to a spatiotemporal grid is constructed in three-dimensional space. According to the hash index table, entities are rendered as three-dimensional spheres located within the corresponding three-dimensional cube. Based on the triplet data, three-dimensional curves are drawn between the three-dimensional spheres of the associated entities to achieve the visualization of the spatiotemporal knowledge graph. Responding to user interaction commands, the current spatiotemporal grid is dynamically switched, and the visualization content is updated.

[0007] Optionally, the step of constructing a hash index table that associates the spatiotemporal grid code with the entity's unique identifier based on the hash key of the spatiotemporal grid code, and storing triple data representing the relationships between entities, includes: The key spatiotemporal attributes in the spatiotemporal knowledge graph conceptual model are converted into the spatiotemporal grid code, and the unique identifiers of entities are stored in the hash index table according to the conversion results; The non-spatiotemporal relationships of entities are stored in the graph dataset of the spatiotemporal knowledge graph concept model in the form of triples.

[0008] Optionally, the step of drawing a three-dimensional curve between three-dimensional spheres of the associated entities based on the triplet data includes: Obtain the unique identifier of the entity within the current spatiotemporal grid from the hash index table, and query the relevant triplet data from the graph dataset based on the unique identifier of the entity; Based on the triplet data, two entity nodes located in the same spatiotemporal grid cell are used to construct a three-dimensional real curve; two entity nodes located in different spatiotemporal grid cells are used to construct a three-dimensional imaginary curve.

[0009] Optionally, the spatiotemporal knowledge graph visualization also includes an initial rendering process; The initial rendering process specifically includes: An initial spatiotemporal grid is determined, and the initial spatiotemporal grid cell is rendered as a three-dimensional cube with a first transparency as the starting point for map visualization. The associated grids of the initial spatiotemporal grid are rendered as three-dimensional cubes with a second transparency. Based on the spatial and temporal encoding of the spatiotemporal grid, all unique identifiers of entities within the initial spatiotemporal grid are retrieved from the hash index table, and rendered using solid 3D spheres of different colors according to the entity type; based on the hash index table and the graph dataset, all entity nodes within the initial spatiotemporal grid and the 3D solid and 3D dashed curves between entities are rendered to form a knowledge graph element string.

[0010] Optionally, the spatial grid division method includes the equidistant division method and the equilatitude and longitude division method; The equidistant division method divides the latitude and longitude directions according to fixed length and width distances; the equidistant latitude and longitude division method divides the latitude and longitude directions according to fixed longitude and latitude intervals.

[0011] Optionally, dynamically switching the currently focused spatiotemporal grid in response to user interaction commands includes: In response to the first interaction command, the target cube selected by the user is set as the new initial spatiotemporal grid; In response to the second interactive command, the 3D cube corresponding to the user-specified mesh code is set as the new initial spatiotemporal mesh; In response to the third interaction command, when it is determined that the entity relationship curve selected by the user is not within the current spatiotemporal grid, a semi-transparent three-dimensional sphere marker is generated in the spatiotemporal grid direction of the entity relationship curve; and further in response to the user's selection operation of the semi-transparent three-dimensional sphere marker, the corresponding spatiotemporal grid is switched to a new initial spatiotemporal grid. In response to the fourth interaction command, the system determines whether the user selects a spatiotemporal grid at an upper or lower scale, calculates the corresponding grid code for the upper or lower scale based on the current grid code, and switches the corresponding grid code to the new initial spatiotemporal grid.

[0012] Optionally, the updated visualization content includes: In response to the fifth interaction command, it is determined that the user has selected a three-dimensional sphere representing an entity in the graph. Taking the entity of the three-dimensional sphere as the central entity, other three-dimensional spheres of entities that are related to it are queried and rendered according to the preset relationship hop number. Among them, related entities located in the same spatiotemporal grid as the central entity are marked with solid lines, and related entities located in different spatiotemporal grids as the central entity are marked with dashed lines. In response to the sixth interaction command, if it is determined that the user has selected a target relationship representing an entity in the graph, then the three-dimensional spheres of the head and tail entities associated with the target relationship will be highlighted.

[0013] Optionally, the dynamic switching of the currently focused spatiotemporal grid further includes: Set up X-axis, Y-axis, and Z-axis navigation bars in the visualization interface; the X-axis, Y-axis, and Z-axis correspond to spatial latitude, spatial longitude, and time dimension, respectively; Adjust the X-axis navigation bar to move the current spatiotemporal grid along the corresponding spatial latitude direction; Adjust the Y-axis navigation bar to move the current spatiotemporal grid along the corresponding spatial longitude direction; Adjust the Z-axis navigation bar to move the current spatiotemporal grid along the time dimension while maintaining its spatial position; the Z-axis navigation bar is also used to place all entities corresponding to the current spatiotemporal grid into the display space of the corresponding three-dimensional cube.

[0014] Secondly, this application discloses a maritime spatiotemporal knowledge graph visualization system, comprising: The encoding module is used to divide the projection plane of Earth's latitude and longitude space into multiple spatial grids and encode them to obtain a multi-scale pyramid structure spatial code. It also divides the time dimension into equal-length time segments and encodes them to obtain a time code. The spatial code and the time code are merged to obtain a spatiotemporal grid code. The spatiotemporal knowledge graph construction module is used to construct a hash index table that associates the spatiotemporal grid code with the entity unique identifier, using the spatiotemporal grid code as a hash key, and to obtain and store triple data representing the relationship between entities. The visualization module is used to construct a three-dimensional cube corresponding to the spatiotemporal grid in three-dimensional space based on a three-dimensional digital earth, render entities as three-dimensional spheres located in the corresponding three-dimensional cube according to the hash index table, and draw three-dimensional curves between the three-dimensional spheres of the associated entities according to the triplet data, so as to realize the visualization of the spatiotemporal knowledge graph. The human-computer interaction module is used to respond to user interaction commands, dynamically switch the current spatiotemporal grid, and update the visualization content.

[0015] Thirdly, this application provides an electronic device, comprising: at least one memory for storing a program; and at least one processor for executing the program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to execute the method described in the first aspect or any possible implementation thereof.

[0016] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0017] Fifthly, this application provides a computer program product that, when run on a processor, causes the processor to perform the method described in the first aspect or any possible implementation thereof.

[0018] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here.

[0019] Overall, the technical solutions conceived in this application have the following beneficial effects compared with the prior art: (1) This application realizes the unified expression and in-depth exploration of multidimensional relationships in the marine spatiotemporal knowledge graph by constructing a data organization model that integrates spatiotemporal grid coding and hash index, a visualization mapping model based on three-dimensional digital earth, and a dynamic interactive exploration mechanism. It not only fully preserves the semantic relationships of traditional knowledge graph entities expressed by three-dimensional curves, but also innovatively presents the distribution, evolution and status of entities in the spatial dimension, time dimension and spatiotemporal joint dimension through the mapping relationship of spatiotemporal grid three-dimensional cubes. In a unified interactive three-dimensional visualization environment, it realizes the integrated cognition and analysis of the spatiotemporal attributes and semantic associations of marine entities.

[0020] (2) This application can realize true spatiotemporal knowledge graph visualization with a simple and beautiful display effect; and supports spatiotemporal grid ranges of different scales, and the grid scale can be customized according to application requirements; and the human-computer interaction of this application is simple and friendly, and easy to understand. Attached Figure Description

[0021] Figure 1 This is one of the flowcharts illustrating the maritime spatiotemporal knowledge graph visualization method provided in the embodiments of this application; Figure 2 This is a schematic diagram of mesh division according to an embodiment of this application; Figure 3 This is a schematic diagram showing the spatiotemporal knowledge graph visualization model according to an embodiment of this application; Figure 4 This is one of the flowcharts illustrating the maritime spatiotemporal knowledge graph visualization method provided in the embodiments of this application; Figure 5 This is a schematic diagram of the structure of the maritime spatiotemporal knowledge graph visualization device provided in the embodiments of this application; Figure 6 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0023] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship; for example, A / B means A or B.

[0024] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0025] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0026] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0027] The embodiments of this application are described below with reference to the accompanying drawings.

[0028] Reference Figure 1 This application provides a method for visualizing maritime spatiotemporal knowledge graphs, including: S101. Divide the projection plane of Earth's latitude and longitude space into multiple spatial grids and encode them to obtain a multi-scale pyramid structure spatial code. Divide it into equal-length time segments in the time dimension and encode them to obtain a time code. Merge the spatial code and the time code to obtain a spatiotemporal grid code. S102. Using the spatiotemporal grid code as a hash key, construct a hash index table that associates the spatiotemporal grid code with the entity's unique identifier, and obtain and store triple data representing the relationship between entities; S103. Based on the three-dimensional digital earth, a three-dimensional cube corresponding to the spatiotemporal grid is constructed in the three-dimensional space. According to the hash index table, the entity is rendered as a three-dimensional sphere located in the corresponding three-dimensional cube. According to the triplet data, a three-dimensional curve is drawn between the three-dimensional spheres of the associated entity to realize the visualization of the spatiotemporal knowledge graph. S104. Responding to user interaction commands, dynamically switch the current spatiotemporal grid and update the visualization content.

[0029] First, using S101 and the Earth's latitude and longitude model as the organizational basis, the projection plane formed by the -180° to -180° longitude space and the -90° to -90° latitude space is divided into several spatial grids. The grid division method can be selected according to needs, such as the equidistant division method and the equidistant latitude and longitude division method. The former divides the latitude and longitude directions according to fixed length and width distances, while the latter divides the latitude and longitude directions according to fixed longitude and latitude ranges. Both division methods have their advantages and disadvantages, and the implementer can choose the division method according to its own needs.

[0030] Furthermore, refer to Figure 2 The grid can be divided into multiple layers and scales according to the needs of the application. For example, the upper layer grid consists of four lower layer grids, and so on, forming a pyramid-shaped grid subdivision structure from bottom to top. Each spatial grid obtained is assigned a unique spatial code.

[0031] Simultaneously, in the time dimension, the time axis is divided into equal-length time segments, and each time segment is assigned a unique time code. The spatial code and the time code together constitute a spatiotemporal grid code, which can uniquely identify a three-dimensional spatiotemporal unit covering a specific sea area within a specific time period.

[0032] Secondly, through S102, an efficient data indexing and organization system is constructed based on the aforementioned spatiotemporal grid encoding. The core is the creation of a hash index table, which uses the spatiotemporal grid encoding as the hash key. During the data preprocessing stage, all entities in the spatiotemporal knowledge graph are traversed, and the key spatiotemporal attributes of each entity are extracted. These continuous spatiotemporal coordinate values ​​are converted into corresponding spatial and temporal codes, thereby determining the spatiotemporal grid to which the entity belongs.

[0033] Subsequently, the entity's unique identifier (such as a UUID) is stored in a hash table entry with the spatiotemporal grid encoding as the key. In this way, the hash index table establishes a fast mapping from spatiotemporal location to the set of entities located there.

[0034] Furthermore, the traditional, non-spatiotemporal semantic relationships between entities continue to be stored as triples in the form of head entity, relation, and tail entity, forming a graph dataset.

[0035] Further, step S103 realizes the visual representation of knowledge in a three-dimensional graphics environment. A three-dimensional digital earth is used as the rendering base and spatial reference frame. According to the partitioning rules of S101, each spatiotemporal grid is geometrically mapped in the three-dimensional scene. Its bottom surface corresponds to the spatial grid range of the geographic surface, and its height corresponds to the length of the time segment represented by that grid, thereby constructing a series of semi-transparent three-dimensional cubes above the surface of the digital earth, each cube representing a spatiotemporal unit.

[0036] During visualization rendering, the spatiotemporal grid to be displayed is determined based on the current view area or user specifications. For each target cube, the hash index table constructed in S102 is queried to obtain a list of unique identifiers for all entities located within that grid. For each entity in the list, a 3D sphere is rendered at the corresponding position on the 3D digital globe as its visual symbol. Different categories of entities can be distinguished by the color and size of the sphere.

[0037] Next, the triplet dataset is queried, and for each relationship that associates two rendered entities, a 3D curve is drawn between the two 3D spheres. By using different curve styles (such as solid and dashed lines, different colors), the type or strength of the relationship can be expressed, thus integrating the traditional knowledge graph relationship network with the spatiotemporal cube framework into the same visual scene, forming an intuitive spatiotemporal knowledge graph.

[0038] Finally, through S104, the transition from static display to dynamic analysis is achieved. Users can interact with the 3D visualization scene in multiple ways, with the core being the dynamic switching of the spatiotemporal grid focused on the current observation and analysis. The focus spatiotemporal grid is redefined through interactive operations, and the query conditions of the hash index table and the rendering content of the 3D scene are updated accordingly, thereby enabling user-led, holistic analysis of the maritime spatiotemporal knowledge graph.

[0039] Optionally, the step of constructing a hash index table that associates the spatiotemporal grid code with the entity's unique identifier based on the hash key of the spatiotemporal grid code, and storing triple data representing the relationships between entities, includes: The key spatiotemporal attributes in the spatiotemporal knowledge graph conceptual model are converted into the spatiotemporal grid code, and the unique identifiers of entities are stored in the hash index table according to the conversion results; The non-spatiotemporal relationships of entities are stored in the graph dataset of the spatiotemporal knowledge graph concept model in the form of triples.

[0040] Specifically, this embodiment uses a HASH table to implement the data index table of the spatiotemporal grid partitioning model. The HASH key is constructed by spatial encoding and time encoding. The storage unit in the HASH table stores the identifier information of the entity. The entity's detailed data can be uniquely located based on the entity's identifier.

[0041] Based on the key spatiotemporal attribute requirements of entities in the spatiotemporal knowledge graph conceptual model, the key spatiotemporal attributes are converted into corresponding spatial and temporal codes according to latitude, longitude, and time. Based on the conversion results, the unique identifier of the entity (a code that can uniquely identify an entity, such as UUID) is stored in the HASH index table. The implementer can retrieve the entity's detailed data from the entity storage through this identifier in the HASH table.

[0042] Non-spatial relationships between entities are still stored using the existing triplet method.

[0043] Optionally, the step of drawing a three-dimensional curve between three-dimensional spheres of the associated entities based on the triplet data includes: Obtain the unique identifier of the entity within the current spatiotemporal grid from the hash index table, and query the relevant triplet data from the graph dataset based on the unique identifier of the entity; Based on the triplet data, two entity nodes located in the same spatiotemporal grid cell are used to construct a three-dimensional real curve; two entity nodes located in different spatiotemporal grid cells are used to construct a three-dimensional imaginary curve.

[0044] Specifically, in this embodiment, a three-dimensional digital earth is used as the base. According to the rules of spatiotemporal grid partitioning, the Earth's surface is divided into several grids using semi-transparent spatial curves. Each grid corresponds to a unit of spatial partitioning. In the Z direction, it is divided into several three-dimensional cubes based on temporal partitioning. Each three-dimensional cube represents a spatiotemporal grid partitioning unit.

[0045] Furthermore, by parsing the HASH index table, all entity IDs in the index table are extracted. Entity nodes are represented by 3D spheres of different colors according to the entity type. Based on the entity ID, the relevant triples are queried in the graph dataset. Based on the second entity ID in the triple, it is determined whether the entity should be drawn in the current spatiotemporal grid subdivision cell. If the second entity is also in this cell, it is drawn in the same way as the first entity, and the relationship between the two is drawn as a 3D solid curve. If the second entity is not in this cell, its spatiotemporal encoding is used to determine which direction it is in of the 3D cube, and a 3D dashed curve is used to connect the entity to the center point of the face.

[0046] Optionally, the spatiotemporal knowledge graph visualization also includes an initial rendering process; The initial rendering process specifically includes: An initial spatiotemporal grid is determined, and the initial spatiotemporal grid cell is rendered as a three-dimensional cube with a first transparency as the starting point for map visualization. The associated grids of the initial spatiotemporal grid are rendered as three-dimensional cubes with a second transparency. Based on the spatial and temporal encoding of the spatiotemporal grid, all unique identifiers of entities within the initial spatiotemporal grid are retrieved from the hash index table, and rendered using solid 3D spheres of different colors according to the entity type; based on the hash index table and the graph dataset, all entity nodes within the initial spatiotemporal grid and the 3D solid and 3D dashed curves between entities are rendered to form a knowledge graph element string.

[0047] Specifically, in this embodiment, an initial spatiotemporal grid is first selected as the starting point for map visualization. For example, the previously visited spatiotemporal grid can be selected as the initial spatiotemporal grid, or the center point of the current map can be specified as the initial spatiotemporal grid.

[0048] Reference Figure 3 , Figure 3 This is a schematic diagram of the spatiotemporal knowledge graph visualization model in this embodiment. The initial spatiotemporal grid is rendered as a semi-transparent 3D cube. It is recommended to use a low-saturation blue as the base color and the transparency should not exceed 0.5. The spatiotemporal grids associated with it in the 6 directions are rendered as 3D cubes with higher transparency (not less than 0.7) and arranged in a wheel-like pattern. The grids are connected by 3D straight line segments to represent their spatial relationships. The remaining cube grids are kept hidden.

[0049] Based on the spatiotemporal encoding of the grid, all entity IDs within the initial grid are retrieved from the HASH index table. According to the entity classification, they are rendered using solid 3D spheres of different colors. The initial position of the sphere can be determined based on the actual latitude and longitude coordinates of the entity. Based on the entity ID, the triple information related to it is obtained from the entity dataset. Based on the triple information, a 3D relationship curve is rendered according to the visualization model requirements to form a knowledge graph element string (similar to a grape bunch structure). The starting entity can be selected manually or randomly. The number of relationship hops is manually selected, with a default of 1 hop.

[0050] Optionally, dynamically switching the currently focused spatiotemporal grid in response to user interaction commands includes: In response to the first interaction command, the target cube selected by the user is set as the new initial spatiotemporal grid; In response to the second interactive command, the 3D cube corresponding to the user-specified mesh code is set as the new initial spatiotemporal mesh; In response to the third interaction command, when it is determined that the entity relationship curve selected by the user is not within the current spatiotemporal grid, a semi-transparent three-dimensional sphere marker is generated in the spatiotemporal grid direction of the entity relationship curve; and further in response to the user's selection operation of the semi-transparent three-dimensional sphere marker, the corresponding spatiotemporal grid is switched to a new initial spatiotemporal grid. In response to the fourth interaction command, the system determines whether the user selects a spatiotemporal grid at an upper or lower scale, calculates the corresponding grid code for the upper or lower scale based on the current grid code, and switches the corresponding grid code to the new initial spatiotemporal grid.

[0051] It should be further noted that during the interaction according to the fourth interaction command, the calculation method for calculating the upper / lower scale from the current grid encoding is determined by the specific grid division method of the implementer. This application does not impose any restrictions, but only provides an example. Optionally, the dynamic switching of the currently focused spatiotemporal grid also includes: Set up X-axis, Y-axis, and Z-axis navigation bars in the visualization interface; the X-axis, Y-axis, and Z-axis correspond to spatial latitude, spatial longitude, and time dimension, respectively; Adjust the X-axis navigation bar to move the current spatiotemporal grid along the corresponding spatial latitude direction; Adjust the Y-axis navigation bar to move the current spatiotemporal grid along the corresponding spatial longitude direction; Adjust the Z-axis navigation bar to move the current spatiotemporal grid along the time dimension while maintaining its spatial position; the Z-axis navigation bar is also used to place all entities corresponding to the current spatiotemporal grid into the display space of the corresponding three-dimensional cube.

[0052] Specifically, in this embodiment, the current grid display mode is adjusted and the grid is switched according to user instructions: If the user clicks on one of the nine surrounding cubes, that cube is set as the initial grid, and the process jumps to the above steps to determine the initial spatiotemporal grid. The initial spatiotemporal grid unit is rendered as a three-dimensional cube with a first transparency as the starting point for map visualization, and the associated grids of the initial spatiotemporal grid are rendered as three-dimensional cubes with a second transparency.

[0053] If the user inputs a specified grid code, the cube corresponding to that grid code is set as the initial grid, and the process jumps to the above steps to determine the initial spatiotemporal grid. The initial spatiotemporal grid unit is rendered as a three-dimensional cube with a first transparency as the starting point for map visualization, and the associated grids of the initial spatiotemporal grid are rendered as three-dimensional cubes with a second transparency.

[0054] If the user selects an entity relationship curve, and the second entity associated with that curve is not within the current spatiotemporal grid, a semi-transparent 3D sphere is drawn in the spatiotemporal direction of that entity, connecting the end of the curve to it. When the user clicks on the 3D sphere, the grid containing the sphere is set as the initial grid, and the process jumps to the steps described above to determine the initial spatiotemporal grid. The initial spatiotemporal grid cell is rendered as a 3D cube with a first transparency level, serving as the starting point for the map visualization. The associated grids of the initial spatiotemporal grid are then rendered as 3D cubes with a second transparency level.

[0055] If the user selects to display the spatiotemporal grid at the upper (lower) scale, the spatiotemporal grid code of the upper (lower) scale is obtained based on the current grid code. The cube corresponding to the grid code is set as the initial grid. The process jumps to the above steps to determine the initial spatiotemporal grid, renders the initial spatiotemporal grid unit as a three-dimensional cube with the first transparency as the starting point for map visualization, and renders the associated grids of the initial spatiotemporal grid as three-dimensional cubes with the second transparency.

[0056] Furthermore, this embodiment also includes X, Y, and Z three-axis navigation bars at the top of the interface. Adjusting the X-axis navigation bar moves the current grid cube in the spatial latitude direction (while keeping the longitude direction unchanged); adjusting the Y-axis navigation bar moves the current grid cube in the spatial longitude direction (while keeping the latitude direction unchanged); adjusting the Z-axis navigation bar moves the current grid cube in the time dimension (keeping the spatial latitude and longitude unchanged). A collapse button is provided on the Z-axis navigation bar. When clicked, all entities within the corresponding spatial grid are placed into the cube display space for display (all time periods).

[0057] Optionally, the updated visualization content includes: In response to the fifth interaction command, it is determined that the user has selected a three-dimensional sphere representing an entity in the graph. Taking the entity of the three-dimensional sphere as the central entity, other three-dimensional spheres of entities that are related to it are queried and rendered according to the preset relationship hop number. Among them, related entities located in the same spatiotemporal grid as the central entity are marked with solid lines, and related entities located in different spatiotemporal grids as the central entity are marked with dashed lines. In response to the sixth interaction command, if it is determined that the user has selected a target relationship representing an entity in the graph, then the three-dimensional spheres of the head and tail entities associated with the target relationship will be highlighted.

[0058] Specifically, in this embodiment, a three-dimensional sphere representing an entity is selected in the atlas. Then, with the entity represented by the sphere as the center and the number of relationship jumps as the parameter, other three-dimensional spheres of entities that are related to it are drawn. The color of the sphere is determined by the entity type. Entities that are not in the same spatiotemporal grid as the current entity are displayed with dashed lines (indicating that there is no spatiotemporal relationship between them and the current entity within the spatiotemporal range represented by the current spatiotemporal grid), while entities that are in the same spatiotemporal grid as the current entity are displayed with solid lines (indicating that there is a spatiotemporal relationship between them and the current entity within the spatiotemporal range represented by the current spatiotemporal grid).

[0059] The properties of the selected entity are displayed in a list of available areas of the interface via a pop-up dialog box.

[0060] Selecting a relationship to represent an entity in the graph will highlight the 3D spheres of the head and tail entities associated with that relationship, and the attributes of the selected relationship will be displayed in a list in a pop-up dialog box.

[0061] Reference Figure 4 , Figure 4 This is a complete flowchart of an embodiment of this application, including the following steps: S1. Construct a spatiotemporal knowledge graph knowledge organization model based on grid partitioning; S11. Divide the spatial grid and generate spatial codes; S12. Divide the time grid and generate time codes; S13. Construct a HASH index table based on spatial encoding and time encoding; S14. Link entity, relationship, and attribute data to the knowledge organization model.

[0062] S2. Construct a spatiotemporal knowledge graph visualization model based on Digital Earth; S3. Obtain initial cube and entity data; S4. Draw the initial grid cube; S5. Draw a 3D diagram of knowledge entities and relationships; S6. Receive human-computer interaction instructions.

[0063] Reference Figure 5 This application discloses a maritime spatiotemporal knowledge graph visualization system, comprising: The encoding module 510 is used to divide the projection plane of the Earth's latitude and longitude space into multiple spatial grids and encode them to obtain a multi-scale pyramid structure spatial code, and to divide it into equal-length time segments in the time dimension and encode them to obtain a time code. The spatial code and the time code are merged to obtain a spatiotemporal grid code. The spatiotemporal knowledge graph construction module 520 is used to construct a hash index table that associates the spatiotemporal grid code with the entity unique identifier, using the spatiotemporal grid code as a hash key value, and to obtain and store triple data representing the relationship between entities. The visualization module 530 is used to construct a three-dimensional cube corresponding to the spatiotemporal grid in three-dimensional space based on a three-dimensional digital earth, render entities as three-dimensional spheres located in the corresponding three-dimensional cube according to the hash index table, and draw three-dimensional curves between the three-dimensional spheres of the associated entities according to the triplet data, so as to realize the visualization of the spatiotemporal knowledge graph. The human-computer interaction module 540 is used to respond to user interaction commands, dynamically switch the current spatiotemporal grid, and update the visualization content.

[0064] Optionally, the step of constructing a hash index table that associates the spatiotemporal grid code with the entity's unique identifier based on the hash key of the spatiotemporal grid code, and storing triple data representing the relationships between entities, includes: The key spatiotemporal attributes in the spatiotemporal knowledge graph conceptual model are converted into the spatiotemporal grid code, and the unique identifiers of entities are stored in the hash index table according to the conversion results; The non-spatiotemporal relationships of entities are stored in the graph dataset of the spatiotemporal knowledge graph concept model in the form of triples.

[0065] Optionally, the step of drawing a three-dimensional curve between three-dimensional spheres of the associated entities based on the triplet data includes: Obtain the unique identifier of the entity within the current spatiotemporal grid from the hash index table, and query the relevant triplet data from the graph dataset based on the unique identifier of the entity; Based on the triplet data, two entity nodes located in the same spatiotemporal grid cell are used to construct a three-dimensional real curve; two entity nodes located in different spatiotemporal grid cells are used to construct a three-dimensional imaginary curve.

[0066] Optionally, the spatiotemporal knowledge graph visualization also includes an initial rendering process; The initial rendering process specifically includes: An initial spatiotemporal grid is determined, and the initial spatiotemporal grid cell is rendered as a three-dimensional cube with a first transparency as the starting point for map visualization. The associated grids of the initial spatiotemporal grid are rendered as three-dimensional cubes with a second transparency. Based on the spatial and temporal encoding of the spatiotemporal grid, all unique identifiers of entities within the initial spatiotemporal grid are retrieved from the hash index table, and rendered using solid 3D spheres of different colors according to the entity type; based on the hash index table and the graph dataset, all entity nodes within the initial spatiotemporal grid and the 3D solid and 3D dashed curves between entities are rendered to form a knowledge graph element string.

[0067] Optionally, the spatial grid division method includes the equidistant division method and the equilatitude and longitude division method; The equidistant division method divides the latitude and longitude directions according to fixed length and width distances; the equidistant latitude and longitude division method divides the latitude and longitude directions according to fixed longitude and latitude ranges.

[0068] Optionally, dynamically switching the currently focused spatiotemporal grid in response to user interaction commands includes: In response to the first interaction command, the target cube selected by the user is set as the new initial spatiotemporal grid; In response to the second interactive command, the 3D cube corresponding to the user-specified mesh code is set as the new initial spatiotemporal mesh; In response to the third interaction command, when it is determined that the entity relationship curve selected by the user is not within the current spatiotemporal grid, a semi-transparent three-dimensional sphere marker is generated in the spatiotemporal grid direction of the entity relationship curve; and further in response to the user's selection operation of the semi-transparent three-dimensional sphere marker, the corresponding spatiotemporal grid is switched to a new initial spatiotemporal grid. In response to the fourth interaction command, the system determines whether the user selects a spatiotemporal grid at an upper or lower scale, calculates the corresponding grid code for the upper or lower scale based on the current grid code, and switches the corresponding grid code to the new initial spatiotemporal grid.

[0069] Optionally, the updated visualization content includes: In response to the fifth interaction command, it is determined that the user has selected a three-dimensional sphere representing an entity in the graph. Taking the entity of the three-dimensional sphere as the central entity, other three-dimensional spheres of entities that are related to it are queried and rendered according to the preset relationship hop number. Among them, related entities located in the same spatiotemporal grid as the central entity are marked with solid lines, and related entities located in different spatiotemporal grids as the central entity are marked with dashed lines. In response to the sixth interaction command, if it is determined that the user has selected a target relationship representing an entity in the graph, then the three-dimensional spheres of the head and tail entities associated with the target relationship will be highlighted.

[0070] Optionally, the dynamic switching of the currently focused spatiotemporal grid further includes: Set up X-axis, Y-axis, and Z-axis navigation bars in the visualization interface; the X-axis, Y-axis, and Z-axis correspond to spatial latitude, spatial longitude, and time dimension, respectively; Adjust the X-axis navigation bar to move the current spatiotemporal grid along the corresponding spatial latitude direction; Adjust the Y-axis navigation bar to move the current spatiotemporal grid along the corresponding spatial longitude direction; Adjust the Z-axis navigation bar to move the current spatiotemporal grid along the time dimension while maintaining its spatial position; the Z-axis navigation bar is also used to place all entities corresponding to the current spatiotemporal grid into the display space of the corresponding three-dimensional cube.

[0071] Reference Figure 6 Based on the methods in the above embodiments, this application provides an electronic device that may include: a processor 610, a communications interface 620, a memory 630, and a communication bus 640. The processor 610, communications interface 620, and memory 630 communicate with each other via the communication bus 640. The processor 610 can call logical instructions in the memory 630 to execute the methods in the above embodiments.

[0072] Furthermore, the logical instructions in the aforementioned memory 630 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.

[0073] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0074] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0075] It is understood that the processor in the embodiments of this application can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor can be a microprocessor or any conventional processor.

[0076] The method steps in this application embodiment can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0077] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0078] It is understood that the various numerical designations used in the embodiments of this application are merely for the convenience of description and are not intended to limit the scope of the embodiments of this application.

[0079] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for visualizing a maritime spatiotemporal knowledge graph, characterized in that, include: The projection plane of Earth's latitude and longitude space is divided into multiple spatial grids and encoded to obtain a multi-scale pyramid structure spatial code. It is also divided into equal-length time segments in the time dimension and encoded to obtain a time code. The spatial code and the time code are merged to obtain a spatiotemporal grid code. Using the spatiotemporal grid code as a hash key, a hash index table is constructed that associates the spatiotemporal grid code with the entity's unique identifier, and triple data representing the relationship between entities is obtained and stored. Based on a three-dimensional digital earth, a three-dimensional cube corresponding to a spatiotemporal grid is constructed in three-dimensional space. According to the hash index table, entities are rendered as three-dimensional spheres located within the corresponding three-dimensional cube. Based on the triplet data, three-dimensional curves are drawn between the three-dimensional spheres of the associated entities to achieve the visualization of the spatiotemporal knowledge graph. Responding to user interaction commands, the current spatiotemporal grid is dynamically switched, and the visualization content is updated.

2. The maritime spatiotemporal knowledge graph visualization method according to claim 1, characterized in that, The step of using the spatiotemporal grid code as a hash key to construct a hash index table associating the spatiotemporal grid code with the entity's unique identifier, and storing triple data representing the relationships between entities, includes: The key spatiotemporal attributes in the spatiotemporal knowledge graph conceptual model are converted into the spatiotemporal grid code, and the unique identifiers of entities are stored in the hash index table according to the conversion results; The non-spatiotemporal relationships of entities are stored in the graph dataset of the spatiotemporal knowledge graph concept model in the form of triples.

3. The maritime spatiotemporal knowledge graph visualization method according to claim 1, characterized in that, The step of drawing a three-dimensional curve between three-dimensional spheres of the associated entities based on the triplet data includes: Obtain the unique identifier of the entity within the current spatiotemporal grid from the hash index table, and query the relevant triplet data from the graph dataset based on the unique identifier of the entity; Based on the triplet data, two entity nodes located in the same spatiotemporal grid cell are used to construct a three-dimensional real curve; two entity nodes located in different spatiotemporal grid cells are used to construct a three-dimensional imaginary curve.

4. The maritime spatiotemporal knowledge graph visualization method according to claim 3, characterized in that, The spatiotemporal knowledge graph visualization also includes an initial rendering process; The initial rendering process specifically includes: An initial spatiotemporal grid is determined, and the initial spatiotemporal grid cell is rendered as a three-dimensional cube with a first transparency as the starting point for map visualization. The associated grids of the initial spatiotemporal grid are rendered as three-dimensional cubes with a second transparency. Based on the spatial and temporal encoding of the spatiotemporal grid, all unique identifiers of entities within the initial spatiotemporal grid are retrieved from the hash index table, and rendered using solid 3D spheres of different colors according to the entity type; based on the hash index table and the graph dataset, all entity nodes within the initial spatiotemporal grid and the 3D solid and 3D dashed curves between entities are rendered to form a knowledge graph element string.

5. The maritime spatiotemporal knowledge graph visualization method according to claim 1, characterized in that, The spatial grid division methods include the equidistant division method and the equilatitude and longitude division method; The equidistant division method divides the latitude and longitude directions according to fixed length and width distances; the equidistant latitude and longitude division method divides the latitude and longitude directions according to fixed longitude and latitude intervals.

6. The maritime spatiotemporal knowledge graph visualization method according to claim 1, characterized in that, The dynamic switching of the currently focused spatiotemporal grid in response to user interaction commands includes: In response to the first interaction command, the target cube selected by the user is set as the new initial spatiotemporal grid; In response to the second interactive command, the 3D cube corresponding to the user-specified mesh code is set as the new initial spatiotemporal mesh; In response to the third interaction command, when it is determined that the entity relationship curve selected by the user is not within the current spatiotemporal grid, a semi-transparent three-dimensional sphere marker is generated in the spatiotemporal grid direction of the entity relationship curve; and further in response to the user's selection operation of the semi-transparent three-dimensional sphere marker, the corresponding spatiotemporal grid is switched to a new initial spatiotemporal grid. In response to the fourth interaction command, the system determines whether the user selects a spatiotemporal grid at an upper or lower scale, calculates the corresponding grid code for the upper or lower scale based on the current grid code, and switches the corresponding grid code to the new initial spatiotemporal grid.

7. The maritime spatiotemporal knowledge graph visualization method according to claim 1, characterized in that, The updated visualization content includes: In response to the fifth interaction command, it is determined that the user has selected a three-dimensional sphere representing an entity in the graph. Taking the entity of the three-dimensional sphere as the central entity, other three-dimensional spheres of entities that are related to it are queried and rendered according to the preset relationship hop number. Among them, related entities located in the same spatiotemporal grid as the central entity are marked with solid lines, and related entities located in different spatiotemporal grids as the central entity are marked with dashed lines. In response to the sixth interaction command, if it is determined that the user has selected a target relationship representing an entity in the graph, then the three-dimensional spheres of the head and tail entities associated with the target relationship will be highlighted.

8. The maritime spatiotemporal knowledge graph visualization method according to claim 1, characterized in that, The dynamic switching of the currently focused spatiotemporal grid also includes: Set up X-axis, Y-axis, and Z-axis navigation bars in the visualization interface; the X-axis, Y-axis, and Z-axis correspond to spatial latitude, spatial longitude, and time dimension, respectively; Adjust the X-axis navigation bar to move the current spatiotemporal grid along the corresponding spatial latitude direction; Adjust the Y-axis navigation bar to move the current spatiotemporal grid along the corresponding spatial longitude direction; Adjust the Z-axis navigation bar to move the current spatiotemporal grid along the time dimension while maintaining its spatial position; the Z-axis navigation bar is also used to place all entities corresponding to the current spatiotemporal grid into the display space of the corresponding three-dimensional cube.

9. A maritime spatiotemporal knowledge graph visualization system, characterized in that, include: The encoding module is used to divide the projection plane of Earth's latitude and longitude space into multiple spatial grids and encode them to obtain a multi-scale pyramid structure spatial code. It also divides the time dimension into equal-length time segments and encodes them to obtain a time code. The spatial code and the time code are merged to obtain a spatiotemporal grid code. The spatiotemporal knowledge graph construction module is used to construct a hash index table that associates the spatiotemporal grid code with the entity unique identifier, using the spatiotemporal grid code as a hash key, and to obtain and store triple data representing the relationship between entities. The visualization module is used to construct a three-dimensional cube corresponding to the spatiotemporal grid in three-dimensional space based on a three-dimensional digital earth, render entities as three-dimensional spheres located in the corresponding three-dimensional cube according to the hash index table, and draw three-dimensional curves between the three-dimensional spheres of the associated entities according to the triplet data, so as to realize the visualization of the spatiotemporal knowledge graph. The human-computer interaction module is used to respond to user interaction commands, dynamically switch the current spatiotemporal grid, and update the visualization content.

10. An electronic device, characterized in that, include: At least one memory for storing computer programs; At least one processor is configured to execute a program stored in the memory, wherein when the program stored in the memory is executed, the processor is configured to perform the method as described in any one of claims 1-8.

Images