A method for generating natural language description of spatial relations in spatial scenes

Abstract

The present invention discloses a method for calculating and storing information describing the relationship between spatial objects from a spatial scene. The method uses grid division to assist in the relationship calculation between three different types of spatial objects in the spatial scene, namely, points, lines and surfaces, and uses a knowledge graph to store the relationship in the spatial scene for subsequent generation of spatial scene description text. Compared with the prior art, the present invention uses a grid to convert the complex spatial calculation between different types of spatial objects into the calculation of the topological relationship, orientation relationship and distance relationship between surface objects, and uses a knowledge graph to structure the storage of the attributes of each spatial object and the relationship between them. The method is clear and efficient, and can have a wide range of practical significance in the field of geographic information, and can provide effective initial technical support for the generation of rich spatial scene description texts, spatial scene reconstruction and other applications.

Description

Technical Field

[0001] The present invention relates to the field of spatial computing and data processing technology in geographic information, and more specifically to a method for extracting knowledge from natural language descriptions related to grid spatial computing. Background Art

[0002] In the information age, due to the rapid development of the computer industry, information technology continues to develop in the main directions of high performance, low cost, ubiquitous computing and intelligence, focusing on the research of "human-centered" intelligent information processing control technology and Chinese information processing based on biological characteristics, natural language and dynamic image understanding. Information based on text or pictures has become the main way to describe the spatial scenes of human production and life. How to make computers have the ability to construct natural language from graphic information like the human brain, and how to make computers have spatial cognitive thinking and natural language description capabilities are the main scientific issues of current geographic information technology.

[0003] In the task of generating text for describing spatial scenes, previous studies have focused more on calculating the relationship between point objects than on comprehensive object descriptions. They rely on the longitude and latitude information of two points to calculate the distance and azimuth between spatial object points, thereby generating a description of the spatial relationship between points of interest in a simple spatial scene based on rules. Although this method is highly accurate, it ignores the spatial relationship between surface objects and line objects in real spatial scenes. At the same time, this method requires a large amount of mathematical calculations, making it complicated to obtain the relationship between spatial objects in spatial scenes.

[0004] Since the calculation results of spatial relationships are isolated spatial objects and spatial relationship information, when performing cluster analysis or text generation on these massive amounts of complex spatial information data, using relational databases to store and process spatial scenes not only requires large-scale computing power support, but also consumes a large amount of memory resources and repeated calculation time each time. The ability to traverse spatial relationships and extract spatial information is very weak. In fact, it is unable to cope with the structured and convenient acquisition of massive amounts of complex spatial information data - a method for calculating and storing relationships between various types of objects in spatial scenes is needed to meet the needs of rapid generation of rich semantics.

[0005] A graph database is a database that uses a graph structure for storage and query. Its key concepts are entities (i.e., point data) and relationships (i.e., edge data). Edges are used to connect vertices together, allowing for fast graph retrieval operations. Using graph-structured data, we can naturally express spatial object entities in the real world and the various spatial relationships between them, and adapt to changing business needs. The flexible graph query language can also maintain a constant time complexity, making it easy to perform cluster analysis or text generation on complex spatial information data.

[0006] Based on the above research status, there are the following problems in the calculation and storage of object relationships in spatial scenes: First, the traditional spatial relationship calculation method cannot support the calculation of massive relationships between multiple spatial object types, making it difficult to fully obtain all spatial relationship information in a spatial scene. Second, the unstructured storage of spatial relationship information is difficult to meet the urgent needs of accurate description text generation, spatial scene reconstruction, etc. Summary of the invention

[0007] The purpose of the present invention is to design a method for generating a natural language description of spatial relations in a spatial scene in response to the deficiencies of the existing technical methods. The method uses a grid to convert spatial calculations between different types of spatial objects into calculations of topological relations, orientation relations, and distance relations between surface objects, and uses a knowledge graph to perform structured storage of the inherent attributes of spatial objects in the spatial scene and the relationship information between them. It has a wide range of practical significance in geographic information research and provides effective initial technical support for applications such as spatial scene description text generation and scene reconstruction.

[0008] The object of the present invention is achieved in that:

[0009] A method for generating a natural language description of a spatial relationship of a spatial scene, the method comprising:

[0010] Select a map segment and parse the basic attribute information of the point object, line object and surface object, including the object unique identification code, object name, object type, location information of the points that make up the current object, etc. In addition, the intersection points need to be calculated for line objects. The name of the intersection point as an intersection in the spatial scene is determined by the number of lines that the intersection point passes through and the number of line endpoints, and the information is saved as a point object to participate in subsequent spatial calculation tasks.

[0011] According to the scale of the target map fragment, the target map fragment is evenly divided into a grid matrix suitable for the current scale, the location information of each grid is determined and the grid is encoded for more accurate spatial calculation;

[0012] According to the grid information and the location information of point, line and surface objects in the spatial scene, the geographic spatial relationship is connected and analyzed, the subordinate relationship of various types of spatial objects in the spatial scene in the grid matrix is ​​determined, and the attribute information of the spatial objects is updated;

[0013] Based on the grid, the topological relationship, orientation relationship and distance relationship between the spatial objects are calculated in sequence to generate spatial information triples; the current operation is repeated until all spatial relationship calculations between any two spatial objects in the target segment are completed to obtain a set of spatial information triples;

[0014] A spatial information knowledge graph database is established, and different types of spatial object entities are constructed in the database. The natural language qualitative spatial relationship expression between any two spatial object entities is confirmed through a set of spatial information triples, and the natural language relationship information expression between each object is established to complete the construction of the spatial information knowledge graph of the fragment.

[0015] Compared with the prior art, the present invention designs a method for calculating object relationships in spatial scenes and constructing a scene description knowledge graph to solve the problem that the spatial scene description text generated by existing machine learning only considers the spatial relationship between point objects and ignores the spatial relationship description of line objects and surface objects, and the traditional method requires a lot of computing power for calculating the spatial relationship between multiple types of spatial objects and cannot be measured in a unified manner. The method uses a grid to convert the spatial calculation between different types of spatial objects into the calculation of the topological relationship, orientation relationship and distance relationship between surface objects, thereby improving the efficiency and accuracy of qualitative spatial calculations. The knowledge graph is used to perform structured storage of the inherent attributes of spatial objects in the spatial scene and the relationship information between them, thereby providing a clear and concise knowledge reserve for the task of generating rich spatial scene description texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The present invention will be further described below with reference to the accompanying drawings and embodiments, in which:

[0017] Figure 1 It is a flowchart of an execution of a method for calculating object relationships in a spatial scene and constructing a scene description knowledge graph disclosed by the present invention; DETAILED DESCRIPTION

[0018] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, specific embodiments of the present invention are now described in detail with reference to the accompanying drawings.

[0019] See attached Figure 1 The present invention performs object relationship calculation in spatial scenes and scene description knowledge graph representation according to the following steps:

[0020] (I) Get map fragments, parse and store point, line and surface object information in the fragments

[0021] According to the map fragment data, the number and attribute information of the point, line, and surface objects in the spatial scene are analyzed as follows:

[0022] The point objects in the spatial scene are represented by a set P = {p1, p2, ..., p n}, where n represents the number of point objects. p i The included attributes are shown in Table 1.

[0023] Table 1 Point object attribute information

[0024] Property name describe point_id Point object unique identification id name Point object name category Point object class location Point object location coordinates

[0025] Line objects are represented by a set L = {L1, L2, ..., L m}, where m represents the number of line objects. L i The included attributes are shown in Table 2, where category includes "intersecting lines", "non-intersecting lines", "circular lines", etc.; the point object position coordinates in the conn attribute are stored in order, and the first point object and the last point object are the two endpoints of the line object.

[0026] Table 2 Line object attribute information

[0027] Property name describe line_id Unique ID of line object name Line object name category Line object category conn A collection of position coordinates of the point objects that make up the line

[0028] The surface object is represented by a set S = {s1, s2, ..., s q}, where q represents the number of face objects. s i The included attributes are shown in Table 3, where the point object information in the conn attribute is stored in order, satisfying the property that the position coordinates of the first point object and the last point object are the same to determine that the object is a closed surface.

[0029] Table 3 Area object attribute information

[0030] Property name describe polygons_id Unique ID of the face object name Area object name category Area object category conn A collection of point object position coordinates that make up the polygon

[0031] In addition to the basic point, line, and surface information above, in order to meet the subsequent natural language description habits of the spatial scene, it is also necessary to calculate the intersection between line objects to determine the intersection information in the scene, and add the relationship between the intersection point and other spatial objects in the scene to the spatial calculation. The attributes contained in the intersection point are shown in Table 4, and the judgment rules of the name attribute are shown in Table 5.

[0032] Table 4 Intersection attribute information

[0033] Property name describe crossing_id Unique identification id of the intersection location Intersection point location coordinates lines_id The ID set of the line objects passing through the intersection lines_count The number of line objects passing through the intersection name Intersection point name

[0034] Table 5 Rules for determining intersection names

[0035]

[0036] (II) Creating a grid based on map fragments

[0037] Spatial data is collected based on the basic scale. Data at different scales have different spatial resolutions and spatial object expression accuracy. According to the scale differences of map fragments, grids of corresponding scales need to be created for more accurate spatial calculations. Assuming the scale size of the current map fragment is scale, the calculation formula for the width of each grid is:

[0038] width=scale×10 -3

[0039] Grid coding and location information determination: After obtaining the grid size, it is necessary to make full use of the range information of the map segment to obtain the location and unique identification code of each grid. NW ,y NW ), southeast corner coordinate (x SE ,y SE ), then each row has grids, each column has For a network, for a grid with coordinates of row i and column j, the unique identification code is calculated as:

[0040]

[0041] Its location information is (x NW +j×width,y NW -i×width)

[0042] 3. Determine the subordinate relationship between spatial objects and grids

[0043] Constructing a grid and counting the grids where each spatial object is located is actually to establish a subordinate relationship between the spatial object and the grid. In the grid attribute table, in addition to recording the basic information of the grid, the unique identification code of the spatial entity in the grid should also be recorded, as shown in Table 6.

[0044] Table 6 Grid attribute table

[0045] Property name describe Grid_ID Grid unique identification code Grid_X Grid Rows Grid_Y Number of grid columns Grid_size Grid Size Grid_loc Grid coordinate information Object_ID The spatial object id in the grid

[0046] (IV) Calculating the relationship between spatial objects based on grids

[0047] Assume that the set of grids where the reference object is located is O = {o1, o2, ..., o i ,...,o |O|}, the set of grids where the main object is located is P = {p1, p2, ..., p j ,...,p |P|}, the total number of spatial objects N, any spatial object needs to be used as a reference object to calculate the topological relationship r_topo, orientation relationship r_pos and distance r_dis relationship between it and other spatial objects in the segment, and store the calculated spatial information in the triple set R = {(O k ,r,P k )}, where k∈[1,N×(N-1)], r=(r_topo,r_pos,r_dis).

[0048] The topological relationship r_topo between the two spatial objects O and P is described by the relationship between the grid sets they are in. First, the eight grid quantities need to be defined and calculated, as shown in Table 7. The calculation model of the eight basic topological relationships of the main object P for the reference object O is shown in Table 8.

[0049] Table 7 Definition and calculation of 8 types of grid numbers

[0050] definition describe Num_O The number of grid cells occupied by O Num_P The number of grids occupied by P Num_OP The number of grids shared by O and P among the grids occupied by O Num_PO The number of grids shared by P and O among the grids occupied by P Num_OP_border The number of grids in O that share the object boundary with P Num_OP_inside The number of meshes in O that share objects with P Num_PO_border The number of grids in P that contain object boundaries in the grids shared by O Num_PO_inside The number of meshes in P that are shared with O and contain internal meshes of the object

[0051] The position relationship r_pos needs to be calculated separately for any main object mesh p j With any reference object mesh O i The orientation relationship is calculated and the weighted average is obtained to obtain a more accurate orientation relationship. The formula is as follows:

[0052]

[0053] Where pos(o i ,p j ) is the main object mesh p j For reference object mesh o i The direction matrix is ​​initialized to a zero matrix:

[0054] Table 8 8 basic topological relationship calculation models of the main object P for the reference object O

[0055]

[0056]

[0057] set up The calculation of each value in the matrix is ​​shown in Table 9

[0058] The distance relation r_dis needs to be calculated separately for any main object mesh p j With any reference object mesh O i The distance relationship is calculated and the weighted average is obtained to obtain a more accurate distance relationship. The formula is as follows:

[0059]

[0060] where dis(o i ,p j ) is the main object mesh p j For reference object mesh o i The Euclidean distance is:

[0061]

[0062] Table 9 Direction matrix calculation table

[0063]

[0064] 5. Establishing a spatial information knowledge graph database

[0065] A spatial information knowledge graph is established for each map segment, and the spatial information knowledge graph should be a fully connected graph. First, the vertices of the spatial information knowledge graph are confirmed, that is, the spatial "entity" objects are all point objects, line objects, surface objects and the intersections of each line object in the current segment. The "relationship" between every two spatial objects is converted into a qualitative expression in natural language by the result calculated in the previous step. More specifically, there are few accurate descriptions of topological relationships r_topo in natural language descriptions, and important topological relationships in space are often implicitly expressed with words such as "beside" and "inside". When the topological relationship is intersecting, "adjacent" / "beside" is often used in natural language to express; when the topological relationship is covering or containing, "outside" is often used in natural language to express; when the topological relationship is covered or contained, "inside" is often used in natural language to express.

[0066] The orientation relationship is determined by the orientation of the maximum value of the r_pos direction matrix obtained above, and the corresponding natural language qualitative orientation expression result is added to the relationship edge weight between the two spatial objects.

[0067] The distance relationship is determined by the map fragment size map_width and the r_dis obtained above. Let The mapping between the quantitative distance and the qualitative expression in natural language is shown in Table 10, and the corresponding natural language qualitative expression result is added to the relationship edge weight between the two spatial objects.

[0068] Table 10 Mapping between quantitative distance and qualitative expressions in natural language

[0069]

[0070] Repeat the current operation until the creation of the orientation and distance "relationship" between spatial objects in the same map fragment is completed in the spatial information knowledge graph database, and a knowledge graph data model containing topological relationships, orientation relationships, and distance relationships expressed in natural language is obtained.

[0071] The above is only a further explanation of the present invention and is not intended to limit the present invention. Any equivalent implementation of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A method for generating a natural language description of spatial relations of a spatial scene, characterized in that: The method includes: Select a map fragment and parse the basic attribute information of the point object, line object and surface object, including the object unique identification code, object name, object type and location information of the points that make up the current object; calculate the intersection points for the line objects, and confirm the name of the intersection point as an intersection in the spatial scene by the number of lines that the intersection point passes through and the number of line endpoints, and save the name as a point object to participate in subsequent spatial calculation tasks; According to the scale of the target map fragment, the target map fragment is evenly divided into a grid matrix suitable for the current scale, the location information of each grid is determined and the grid is encoded for spatial calculation; According to the grid information and the location information of point, line and surface objects in the spatial scene, the geographic spatial relationship is connected and analyzed, the subordinate relationship of various types of spatial objects in the spatial scene in the grid matrix is ​​determined, and the attribute information of the spatial objects is updated; Based on the grid, the topological relationship, orientation relationship and distance relationship between the spatial objects are calculated in sequence to generate spatial information triples; the current operation is repeated until all spatial relationship calculations between any two spatial objects in the target segment are completed to obtain a set of spatial information triples; Establish a spatial information knowledge graph database, construct different types of spatial object entities in the database, and confirm the natural language qualitative spatial relationship expression between any two spatial object entities through the spatial information triple set, and establish the natural language relationship information expression between each object to complete the construction of the spatial information knowledge graph; According to the map fragment data, the number and attribute information of the point, line, and surface objects in the spatial scene are analyzed as follows: The point objects in the space scene are represented by a set P = {p1, p2, ..., p n }, where n represents the number of point objects; Line objects are represented by a set L = {L1, L2, ..., L m }, where m represents the number of line objects. L i Among the included attributes, category includes "intersecting lines", "non-intersecting lines", and "circular lines". The point object position coordinates in the conn attribute are stored in order, and the first point object and the last point object are the two endpoints of the line object. The surface object is represented by a set S = {s1, s2, ..., s q }, where q represents the number of face objects. s i Among the included attributes, the point object information in the conn attribute is stored in order, satisfying the property that the position coordinates of the first point object and the last point object are the same to determine that the object is a closed surface; In addition to the basic point, line, and surface information above, in order to meet the subsequent natural language description habits of spatial scenes, it is also necessary to calculate the intersection between line objects to determine the intersection information in the scene, and add the relationship between the intersection points and other spatial objects in the scene into the spatial calculation; Spatial data is collected based on the basic scale. Data at different scales have different spatial resolutions and spatial object expression accuracy. According to the scale differences of map fragments, grids of corresponding scales need to be created for more accurate spatial calculations. Suppose the scale size of the current map fragment is scale, and the calculation formula for the width of each grid is: width=scale×10 -3 Grid coding and location information determination: After obtaining the grid size, it is necessary to make full use of the range information of the map fragment to obtain the location and unique identification code of each grid; set the northwest corner coordinates of the map fragment boundary (x NW ,y NW ), southeast corner coordinate (x SE ,y SE ), then each row has grids, each column has For a network, for a grid with coordinates of row i and column j, the unique identification code is calculated as: Its location information is (x NW +j×width,y NW -i×width) Construct a grid and count the grids where each spatial object is located, and establish the subordinate relationship between the spatial object and the grid; in the grid attribute table, in addition to recording the basic information of the grid, the unique identification code of the spatial entity in the grid should also be recorded; Assume that the set of grids where the reference object is located is O = {o1, o2, ..., o i ,...,o |O| }, the set of grids where the main object is located is P = {p1, p2, ..., p j ,...,p |P| }, the total number of spatial objects N, any spatial object needs to be used as a reference object to calculate the topological relationship r_topo, orientation relationship r_pos and distance r_dis relationship between it and other spatial objects in the segment, and store the calculated spatial information in the triple set R = {(O k ,r,P k )}, where k∈[1,N×(N-1)], r=(r_topo,r_pos,r_dis); The topological relationship r_topo between two spatial objects O and P is described by the relationship between the grid sets they are in. Eight types of grid quantities are defined and calculated; The position relationship r_pos needs to be calculated separately for any main object mesh p j With any reference object mesh O i The orientation relationship is calculated and the weighted average is obtained to obtain a more accurate orientation relationship; the formula is as follows: Where pos(o i ,p j ) is the main object mesh p j For reference object mesh o i The direction matrix is ​​initialized to a zero matrix: set up The distance relation r_dis needs to be calculated separately for any main object mesh p j With any reference object mesh O i The distance relationship is calculated and the weighted average is obtained to obtain a more accurate distance relationship; the formula is as follows: where dis(o i ,p j ) is the main object mesh p j For reference object mesh o i The Euclidean distance is: A spatial information knowledge graph is established for each map segment, and the spatial information knowledge graph is a fully connected graph. First, the vertices of the spatial information knowledge graph are confirmed, that is, the spatial "entity" objects are all point objects, line objects, surface objects and the intersection points of each line object in the current segment. The "relationship" between each two spatial objects is converted into a qualitative expression in natural language by the result calculated in the previous step. There is little accurate description of the topological relationship r_topo in the natural language description. When the topological relationship is intersecting, it is expressed in natural language as "adjacent" / "next to"; when the topological relationship is covering or containing, it is expressed in natural language as "outside"; when the topological relationship is covered or contained, it is expressed in natural language as "inside"; The orientation relationship is determined by the orientation of the maximum value of the r_pos direction matrix, and the corresponding natural language qualitative orientation expression result is added to the relationship edge weight between the two spatial objects; The distance relationship is determined by the map fragment size map_width and the r_dis obtained above. Let Then the mapping between the quantitative distance and the qualitative expression in natural language is performed, and the corresponding qualitative expression result in natural language is added to the edge weight of the relationship between the two spatial objects; Repeat the process until the creation of the orientation and distance "relationship" between spatial objects in the same map segment is completed in the spatial information knowledge graph database, and a knowledge graph data model containing topological relationships, orientation relationships, and distance relationships expressed in natural language is obtained.

Images