Urban resource prediction method and system based on multimodal urban knowledge graph

Abstract

The present invention provides a method and system for predicting urban resources based on a multimodal urban knowledge graph. The method includes: obtaining multi-source urban spatiotemporal data, performing ontology abstraction based on the multi-source urban spatiotemporal data, and determining an ontology set corresponding to basic urban elements; performing entity matching based on the multi-source urban spatiotemporal data and the ontology set, and determining an entity object set corresponding to the ontology; performing relationship construction based on the multi-source urban spatiotemporal data and the ontology set, and determining an association relationship set between ontologies; performing knowledge fusion based on the entity object set, the association relationship set, and the multi-source urban spatiotemporal data, and obtaining a corresponding multimodal urban knowledge graph. The method provided by the present invention can be quickly matched and fused into the urban knowledge graph structure based on the multi-source urban spatiotemporal data, and the semantic relationship of the urban spatiotemporal data can be determined, which effectively improves the efficiency and accuracy of urban resource prediction analysis based on the multi-modal urban knowledge graph.

Description

Technical Field

[0001] The present invention relates to the cross-technical field of urban computing and knowledge graphs, and in particular to a method and system for predicting urban resources based on a multimodal urban knowledge graph. In addition, the present invention also relates to an electronic device and a processor-readable storage medium. Background Art

[0002] The Knowledge Graph is a structured identification and storage of factual knowledge in the real world in the form of a graph. Its main components are entities and relations. Entities correspond to objective things, that is, nodes on the graph, and relations correspond to the semantic associations between things, that is, edges on the graph. Among them, an objective fact corresponds to a triple (h, r, t) on the knowledge graph, where h and t are the head entity and the tail entity respectively, and r is the relationship between the head and tail entities.

[0003] Urban computing refers to the use of massive multi-source urban spatiotemporal data collected in urban spatiotemporal scenarios to perform tasks such as analysis, prediction, simulation, and generation to realize smart cities. For example, the historical flow of people in each block of the city and the distribution of points of interest (POI) in the block can be used to predict the flow of people in the block at a given time in the future; or the use of automated machine learning methods can be used to predict the road driving conditions at the next moment based on the historical traffic flow and speed of the road. However, the main methods of urban computing currently often rely on a single type of urban spatiotemporal data or a simple fusion of several types of urban spatiotemporal data for model training and evaluation, failing to fully explore the semantic knowledge in massive multi-source urban spatiotemporal data and lacking effective knowledge representation. Using knowledge graph technology to organize massive multi-source urban spatiotemporal data into a multimodal urban knowledge graph and capture the complex associations between multi-source data through different semantic relationships has great potential to assist knowledge-driven urban resource prediction and analysis tasks. Therefore, how to design an efficient and accurate urban resource prediction and analysis solution based on multimodal urban knowledge graphs based on the combination of urban computing and knowledge graphs has become a problem that needs to be solved urgently. Summary of the invention

[0004] To this end, the present invention provides an urban resource prediction method and system based on a multimodal urban knowledge graph to solve the defect that the urban resource analysis scheme based on the traditional knowledge graph in the prior art has high limitations, resulting in poor accuracy of urban resources.

[0005] In a first aspect, the present invention provides an urban resource prediction method based on a multimodal urban knowledge graph, comprising:

[0006] Acquire multi-source urban spatiotemporal data, perform ontology abstraction based on the multi-source urban spatiotemporal data, and determine an ontology set corresponding to basic elements of the city;

[0007] Perform entity matching based on the multi-source urban spatiotemporal data and the ontology set to determine a set of entity objects corresponding to the ontology;

[0008] Building relationships based on the multi-source urban spatiotemporal data and the ontology set, and determining a set of association relationships between ontologies;

[0009] Performing knowledge fusion based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data to obtain a corresponding multimodal urban knowledge graph;

[0010] Based on the multimodal urban knowledge graph, the urban resources of the target object in the target area are analyzed, and the urban resource analysis results corresponding to the target object in the target area are output.

[0011] Furthermore, before performing ontology abstraction based on the multi-source urban spatiotemporal data and determining the ontology set corresponding to the basic elements of the city, it also includes: determining the structure of the urban knowledge graph based on the graph pattern, and determining the types of entities in the urban knowledge graph and the relationships between various ontologies;

[0012] The ontology abstraction based on the multi-source urban spatiotemporal data to determine the ontology set corresponding to the basic elements of the city specifically includes: performing ontology abstraction from the multi-source urban spatiotemporal data based on the type of the entity and the relationship between the various ontologies to obtain the ontology set corresponding to the basic elements of the city; the ontology set includes a thing ontology, a space ontology and a time ontology; wherein the space ontology includes a region ontology, a business district ontology and a point of interest ontology; the thing ontology includes a user ontology, a brand ontology and corresponding attribute ontology; the time ontology includes timestamp information of different scales and time concept information corresponding to specific semantics; and the attribute ontology corresponds to the attribute information of each ontology.

[0013] Further, entity matching is performed based on the multi-source urban spatiotemporal data and the ontology set to determine an entity object set corresponding to the ontology, specifically including:

[0014] Perform entity matching based on the multi-source urban spatiotemporal data and the ontology set to determine an entity set; perform attribute enhancement based on the entity set to determine corresponding entity attributes; obtain the entity object set based on the entity set and the entity attributes;

[0015] Among them, the entity object set includes the entity set and the entity attributes; the entity set includes the thing entity corresponding to the thing entity, the space entity corresponding to the space entity and the time entity corresponding to the time entity; the entity attributes correspond to the attribute entity, and the entity attributes include holiday information, location area information, activity level information, user portrait information, category brand information, street view remote sensing information and user portrait information.

[0016] Furthermore, the relationship building based on the multi-source urban spatiotemporal data and the ontology set to determine the set of association relationships between ontologies specifically includes: building relationships based on the multi-source urban spatiotemporal data and the ontology set to determine the physical level relationships of ontologies; and performing relationship enhancement processing based on the physical level relationships to determine the semantic level relationships of ontologies; wherein the set of association relationships includes the physical level relationships and the semantic level relationships; the physical level relationships include the positional relationships, temporal relationships and sequence relationships of ontologies; and the semantic level relationships include the attribute relationships, functional relationships, transfer relationships, competitive relationships, causal relationships and co-occurrence relationships of ontologies.

[0017] Furthermore, knowledge fusion is performed based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data to obtain a corresponding multimodal urban knowledge graph, which specifically includes:

[0018] Abstracting the multi-source urban spatiotemporal data into several layers, and determining the ontology corresponding to each layer;

[0019] The multi-source urban spatiotemporal data are analyzed for association relationships, with the relationship mining between entities and the relationship mining between ontologies being carried out, and the instantiation matching process from ontology to entity is realized, knowledge fusion is performed, the corresponding relationship facts are extracted, and the construction of the urban knowledge graph is completed to obtain the corresponding multimodal urban knowledge graph.

[0020] Furthermore, the relationship mining between entities includes:

[0021] Determine the spatial relationship between entities based on their longitude and latitude location information; determine the flow of people transfer relationship between regional entities through massive flow of people data; determine the functional similarity relationship based on the distribution of points of interest in the region; determine the visit co-occurrence relationship based on user access data and time window control; determine the relevant brand information based on the preset Chinese knowledge base, and determine the corresponding category brand relationship according to the category to which the brand belongs; determine the location relationship between brand entities and regional entities based on existing site selection data; determine the corresponding semantic relationship based on the attribute mining, matching and processing of entities; determine the user's residence relationship, work place relationship and corresponding visit relationship based on the user's movement trajectory and stay time point; match the geographic information of street view and remote sensing image data with the longitude and latitude boundaries of the region to mine cross-modal relationship facts.

[0022] In a second aspect, the present invention provides an urban resource prediction system based on a multimodal urban knowledge graph, comprising:

[0023] An ontology determination unit, used to obtain multi-source urban spatiotemporal data, perform ontology abstraction based on the multi-source urban spatiotemporal data, and determine an ontology set corresponding to basic elements of the city;

[0024] An entity object determination unit, configured to perform entity matching based on the multi-source urban spatiotemporal data and the ontology set, and determine an entity object set corresponding to the ontology;

[0025] An association relationship determination unit, configured to construct relationships based on the multi-source urban spatiotemporal data and the ontology set, and determine an association relationship set between ontologies;

[0026] A city knowledge graph determination unit, configured to perform knowledge fusion based on the entity object set, the association relationship set and the multi-source city spatiotemporal data to obtain a corresponding multimodal city knowledge graph;

[0027] The urban resource analysis unit is used to analyze the urban resources of the target object in the target area based on the multimodal urban knowledge graph, and output the urban resource analysis results corresponding to the target object in the target area.

[0028] Furthermore, before performing ontology abstraction based on the multi-source urban spatiotemporal data and determining the ontology set corresponding to the basic elements of the city, it also includes: determining the structure of the urban knowledge graph based on the graph pattern, and determining the types of entities in the urban knowledge graph and the relationships between various ontologies;

[0029] The ontology determination unit is specifically used to: perform ontology abstraction from the multi-source urban spatiotemporal data based on the type of the entity and the relationship between the various ontologies to obtain an ontology set corresponding to the basic elements of the city; the ontology set includes a thing ontology, a space ontology and a time ontology; wherein the space ontology includes a region ontology, a business district ontology and a point of interest ontology; the thing ontology includes a user ontology, a brand ontology and corresponding attribute ontology; the time ontology includes timestamp information of different scales and time concept information corresponding to specific semantics; the attribute ontology corresponds to the attribute information of each ontology.

[0030] Furthermore, the entity object determination unit is specifically used to:

[0031] Perform entity matching based on the multi-source urban spatiotemporal data and the ontology set to determine an entity set; perform attribute enhancement based on the entity set to determine corresponding entity attributes; obtain the entity object set based on the entity set and the entity attributes;

[0032] Among them, the entity object set includes the entity set and the entity attributes; the entity set includes the thing entity corresponding to the thing entity, the space entity corresponding to the space entity and the time entity corresponding to the time entity; the entity attributes correspond to the attribute entity, and the entity attributes include holiday information, location area information, activity level information, user portrait information, category brand information, street view remote sensing information and user portrait information.

[0033] Furthermore, the association relationship determination unit is specifically used to: construct relationships based on the multi-source urban spatiotemporal data and the ontology set to determine the physical level relationships of the ontology; and perform relationship enhancement processing based on the physical level relationships to determine the semantic level relationships of the ontology; wherein the association relationship set includes the physical level relationships and the semantic level relationships; the physical level relationships include the positional relationships, temporal relationships and sequence relationships of the ontology; and the semantic level relationships include the attribute relationships, functional relationships, transfer relationships, competitive relationships, causal relationships and co-occurrence relationships of the ontology.

[0034] Furthermore, the city knowledge graph determination unit is specifically used to:

[0035] Abstracting the multi-source urban spatiotemporal data into several layers, and determining the ontology corresponding to each layer;

[0036] The multi-source urban spatiotemporal data are analyzed for association relationships, with the relationship mining between entities and the relationship mining between ontologies being carried out, and the instantiation matching process from ontology to entity is realized, knowledge fusion is performed, the corresponding relationship facts are extracted, and the construction of the urban knowledge graph is completed to obtain the corresponding multimodal urban knowledge graph.

[0037] Furthermore, the relationship mining between entities includes:

[0038] Determine the spatial relationship between entities based on their longitude and latitude location information; determine the flow of people transfer relationship between regional entities through massive flow of people data; determine the functional similarity relationship based on the distribution of points of interest in the region; determine the visit co-occurrence relationship based on user access data and time window control; determine the relevant brand information based on the preset Chinese knowledge base, and determine the corresponding category brand relationship according to the category to which the brand belongs; determine the location relationship between brand entities and regional entities based on existing site selection data; determine the corresponding semantic relationship based on the attribute mining, matching and processing of entities; determine the user's residence relationship, work place relationship and corresponding visit relationship based on the user's movement trajectory and stay time point; match the geographic information of street view and remote sensing image data with the longitude and latitude boundaries of the region to mine cross-modal relationship facts.

[0039] In a third aspect, the present invention also provides an electronic device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the urban resource prediction method based on the multimodal urban knowledge graph as described in any one of the above are implemented.

[0040] In a fourth aspect, the present invention further provides a processor-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the urban resource prediction method based on the multimodal urban knowledge graph as described in any one of the above are implemented.

[0041] The urban resource prediction method based on the multimodal urban knowledge graph provided by the present invention distills knowledge from massive multi-source urban spatiotemporal data, first abstracts an ontology set with urban basic elements as the core, then matches specific entity objects from the multi-source urban spatiotemporal data, further defines the corresponding association relationship set by mining the semantic association relationship between the ontologies, and finally performs knowledge fusion based on the matched entity objects, the constructed association relationship and the multi-source urban spatiotemporal data to complete the construction of the multi-source urban knowledge graph. It can quickly match and fuse the multi-source urban spatiotemporal data into the ontology and entities of the urban knowledge graph structure, and mine the semantic relationship of the urban spatiotemporal data, effectively improving the efficiency and accuracy of urban resource prediction analysis based on the multimodal urban knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flowchart of a method for predicting urban resources based on a multimodal urban knowledge graph provided by an embodiment of the present invention;

[0044] Figure 2 It is a complete process diagram of urban resource prediction based on multimodal urban knowledge graph provided by an embodiment of the present invention;

[0045] Figure 3 It is a schematic diagram of determining a time entity provided by an embodiment of the present invention;

[0046] Figure 4 It is a schematic diagram of a construction mode of a multimodal urban knowledge graph provided by an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of the road network division of City A provided by an embodiment of the present invention;

[0048] Figure 6 This is a schematic diagram of a region entity within the Fifth Ring Road of City A provided by an embodiment of the present invention;

[0049] Figure 7 is a schematic diagram of a business district entity in city A provided by an embodiment of the present invention;

[0050] Figure 8 It is a schematic diagram of the service radius and service area of ​​a single business district entity provided by an embodiment of the present invention;

[0051] Fig. 9 is a schematic diagram of the distribution of points of interest in city A provided by an embodiment of the present invention;

[0052] Fig.10 is a schematic diagram of the boundary and neighbor relationship facts provided by an embodiment of the present invention;

[0053] Fig.11 is a schematic diagram of the positional relationship between the area and the point of interest provided by an embodiment of the present invention;

[0054] Fig.12 is a schematic diagram of a remote sensing image entity corresponding to a regional entity provided by an embodiment of the present invention;

[0055] Fig.13 It is a structural schematic diagram of an urban resource prediction system based on a multimodal urban knowledge graph provided by an embodiment of the present invention;

[0056] Fig.14 It is a schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] Based on the urban resource prediction method based on the multimodal urban knowledge graph of the present invention, the following describes in detail an embodiment of the method. Figure 1 As shown, it is a flow chart of the urban resource prediction method based on the multimodal urban knowledge graph provided by an embodiment of the present invention, and the specific implementation process includes the following steps:

[0059] Step 101: Acquire multi-source urban spatiotemporal data, perform ontology abstraction based on the multi-source urban spatiotemporal data, and determine an ontology set corresponding to basic elements of the city.

[0060] In an embodiment of the present invention, the structure of the urban knowledge graph can be determined based on the graph pattern, and the types of entities in the urban knowledge graph and the relationships between various ontologies can be determined; based on the types of the entities and the relationships between the various ontologies, ontology abstraction is performed from the multi-source urban spatiotemporal data to obtain an ontology set corresponding to the basic elements of the city; the ontology set includes thing ontology, space ontology and time ontology.

[0061] Among them, the spatial ontology includes regional ontology, business district ontology and point of interest ontology; the thing ontology includes user ontology, brand ontology and corresponding attribute ontology; the time ontology includes timestamp information of different scales and time concept information corresponding to specific semantics; the attribute ontology corresponds to the attribute information of each ontology.

[0062] Specifically, firstly, the structure of the urban knowledge graph is determined based on the graph schema, that is, the structure of the urban knowledge graph is defined from a higher level, and the types of entities in the urban knowledge graph, namely, the ontology, and the relationship between various ontologies are determined. In the actual implementation process, through the abstraction of massive multi-source urban spatiotemporal data, the present invention clarifies the ontologies such as time, region, business area (Ba), point of interest (POI), brand, user, and attribute. Among them, the attribute ontology covers various attributes that other ontologies may have. The ontology of the urban knowledge graph extracted by the present invention is described in detail below. The time ontology is used to describe time information of different scales, including: commonly used time units such as year, month, day, hour, minute, second, and time concept information with specific semantics such as working day, rest day, and holiday. The regional ontology is used to describe a finer-grained division in the urban space, which can be divided according to different standards, such as division according to road network, division according to grid points, etc., aiming to reflect smaller-scale units in space. The business district ontology is used to describe the space formed by the clustering of economic and social activities within the city, focusing on the space formed by human production and living behaviors but without clear physical boundaries. The point of interest ontology: describes things such as infrastructure, buildings, and shops in the city. It can be a school, a hospital, a shop, etc. The brand ontology: describes the unified concept of a certain type of product or product series, such as A catering brand and B catering brand are both catering brands, rather than physical things in the real world. In addition, corporate organizations also belong to the category of brand ontology, such as C Bank, D Company, etc. are all broad brands. The user ontology is used to describe the main activity elements in the city. In the present invention, the user ontology refers to people, cars, etc.; the attribute ontology is used to describe the possible attributes corresponding to the above-mentioned ontology, such as the age, gender, etc. that the user ontology may have, and the boundary latitude and longitude and area that the regional ontology may involve. It should be noted that the attribute ontology will contain more multimodal information, such as visual modal attributes such as street views and remote sensing images, which are not specifically limited here.

[0063] Step 102: Perform entity matching based on the multi-source urban spatiotemporal data and the ontology set to determine a set of entity objects corresponding to the ontology.

[0064] Specifically, entity matching can be performed based on the multi-source urban spatiotemporal data and the ontology set to determine the entity set, and attribute enhancement can be performed based on the entity set to determine the corresponding entity attributes. Among them, the entity object set includes the entity set and the entity attributes; the entity set includes thing entities corresponding to the thing ontology, space entities corresponding to the space ontology, and time entities corresponding to the time ontology; the entity attributes correspond to the attribute ontology, and the entity attributes include holiday information, location area information, activity level information, user portrait information, category brand information, street view remote sensing information and user portrait information. It should be noted that after abstracting the ontology required for the urban knowledge graph based on preset rules, the present invention further performs entity matching from the multi-source urban spatiotemporal data according to the ontology, and instantiates from the concept class ontology to the entity referring to the specific thing, so as to complete the construction of the corresponding node on the urban knowledge graph.

[0065] The entities referring to specific things in the city knowledge graph matched by the present invention are described below. Regarding the time entity: According to the ontology definition, in the actual implementation process of the present invention, the common timestamp in the city spatiotemporal data can be used as an entity, and it can be decomposed into basic time elements, that is, each year, month, day, hour, etc. are time entities on the city knowledge graph. Figure 3is an example of a matched time entity. For the timestamp of 9:00 a.m. on April 27, 2021, it is used as an entity in the city knowledge graph, and it is also associated with the corresponding year entity "2021", the month entity "April", the day entity "27", the hour entity "9 o'clock" and the semantic entity "working day"; similarly, the timestamp of 9:00 a.m. on April 28, 2021 also corresponds to an entity and is associated with the corresponding time basic element entity. For the regional entity: In the actual implementation process of the present invention, the city can be divided according to a certain level of road network, that is, each closed regular geometric area rated by the road network is a regional entity on the urban knowledge graph. Each regional entity contains numerical attribute information such as the regional ID, the longitude and latitude sequence corresponding to the regional boundary, and the regional area. For the business district entity: In the actual implementation process of the present invention, it is proposed to match the business district entity on the urban knowledge graph according to the urban business district divided by the mobile Internet platform (including expert knowledge). For example, "F Business District" is a business district in City A, which gathers many shopping malls and catering stores. Each business district entity contains numerical attribute information such as the business district ID, the business district name, and the longitude and latitude of the center of the business district. Regarding the point of interest entity: In the actual implementation process, the present invention can match all buildings, infrastructure, stores, etc. in the city as point of interest entities. In particular, the present invention subdivides the point of interest entity into point point of interest, surface point of interest, and volume point of interest entity. The point point of interest entity is a point on the map, the surface point of interest entity corresponds to a surface range on the map, and may contain many other point of interest entities, while the volume point of interest entity represents a point of interest with a three-dimensional structure and other points of interest inside. For example, the entity "H Restaurant" is a point point of interest entity, the entity "K University" is a surface point of interest entity, which includes the entire campus, and the entity "L Building" is a body point of interest entity, and each floor inside will have other point of interest entities, etc. Regarding the brand entity: In the actual implementation process, the present invention can match chain store brands in the city and organizations with multiple affiliated points of interest as brand entities. For example, the catering brands H restaurant and M restaurant are both brand entities on the city knowledge graph. Unlike the point of interest entities, brand entities do not have physical locations and are only conceptual entities on the knowledge graph. In addition, it should be noted that entities such as "K University", "K University Affiliated Hospital", "K University Affiliated Middle School", "K University Press" and so on all belong to the same organizational unit, namely the "K University" organization (note that "K University" here refers to a conceptual organization, which is different from the point of interest entity "K University"). Regarding the user entity: in the actual implementation process, the present invention can match people and cars in the city as user entities. Each individual person and car is an entity on the city knowledge graph, and has its own attributes and attribute values; regarding attribute entities: in the actual implementation process, the present invention can match attributes shared by other types of entities as entities on the city knowledge graph.For example, entities of points of interest and brands have category attributes. The present invention divides the corresponding categories into a three-level category system, corresponding to coarse-grained, medium-grained and fine-grained categories, such as the coarse-grained category entity "catering", the medium-grained category entity "Chinese food" and the fine-grained category entity "V local cuisine". Furthermore, the present invention matches street view entities and remote sensing image entities for each regional entity. Specifically, the street view picture corresponding to the boundary of a regional entity is regarded as an entity on the city knowledge graph, describing the infrastructure and buildings of the region; the remote sensing image corresponding to the entire regional entity is regarded as an entity on the city knowledge graph, describing the land use distribution and other information of the region. In addition, the present invention locates events occurring in the city to specific points of interest or regions, describes the events in text and other ways, and treats the events as entities on the city knowledge graph.

[0066] Based on the above entity matching process, the present invention matches concepts, texts, images and other data to entities in the urban knowledge graph according to preset rules, forming a multimodal set of entity objects in the urban knowledge graph.

[0067] Step 103: Building relationships based on the multi-source urban spatiotemporal data and the ontology set, and determining a set of association relationships between ontologies.

[0068] Specifically, the relationship can be constructed based on the multi-source urban spatiotemporal data and the ontology set to determine the physical level relationship of the ontology, and the relationship enhancement processing can be performed based on the physical level relationship to determine the semantic level relationship of the ontology. The association relationship set includes the physical level relationship and the semantic level relationship; the physical level relationship includes the position relationship, time sequence relationship and position sequence relationship of the ontology; the semantic level relationship includes the attribute relationship, function relationship, transfer relationship, competition relationship, causal relationship and co-occurrence relationship of the ontology.

[0069] In this step, the present invention further mines and constructs relationships from massive multi-source urban spatiotemporal data, specifically including spatial relationships, transfer relationships, functional relationships, competitive relationships, attribute relationships, semantically related relationships, user relationships, and cross-modal relationships between entities, which can be specifically expressed as the relationships in the following table:

[0070] Table 1 Construction of multimodal urban knowledge graph relationships

[0071]

[0072]

[0073] It should be noted that the multimodal urban knowledge graph described in the present invention is not limited to binary relationship facts, that is, it is not limited to only involving head entities and tail entities. For multivariate relationships such as human flow transfer, visits and events, the present invention simultaneously considers multivariate relationships between multiple entities, which will not be described in detail here.

[0074] Step 104: Perform knowledge fusion based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data to obtain a corresponding multimodal urban knowledge graph.

[0075] In an embodiment of the present invention, the multi-source urban spatiotemporal data can be abstracted into several layers, and the ontology corresponding to each layer can be determined; the multi-source urban spatiotemporal data can be analyzed for association relationships, and the relationship mining between entities and the relationship mining between ontologies can be carried out, and the instantiation matching process from ontology to entity can be realized, knowledge fusion can be performed, and the corresponding relationship facts can be extracted to complete the construction of the urban knowledge graph, so as to obtain the corresponding multimodal urban knowledge graph. Among them, the relationship mining between entities includes: determining the spatial relationship between them based on the latitude and longitude location information of the entities; determining the flow of people transfer relationship between regional entities through massive flow of people data; determining the functional similarity relationship based on the distribution of points of interest in the region; determining the visit co-occurrence relationship based on user access data and time window control; determining the relevant brand information based on the preset Chinese knowledge base, and determining the corresponding category brand relationship according to the category to which the brand belongs; determining the location relationship between brand entities and regional entities based on existing site selection data; determining the corresponding semantic relationship based on the attribute mining, matching and processing of the entities; determining the user's residence relationship, work place relationship and corresponding visit relationship based on the user's movement trajectory and stay time point; mining cross-modal relationship facts based on the matching of the geographic information of street view and remote sensing image data with the longitude and latitude boundaries of the region.

[0076] Specifically, based on the above ontology abstraction, entity matching and relationship construction process, the present invention proposes the following Figure 4The schematic diagram of the urban knowledge graph construction model is shown in Figure 1. By abstracting the massive multi-source urban spatiotemporal data into several layers (such as Brand layer, POI layer, Businessarea layer, Region layer, etc.), clarifying the ontology corresponding to each layer, and performing the association analysis of the data based on the preset mining algorithm to mine the relationship within and between the ontology, and realizing the instantiation matching process from the ontology to the entity, according to the preset rules, through the data matching and statistical analysis process, the knowledge fusion is realized, the corresponding relationship facts are extracted, and the construction of the multimodal urban knowledge graph is completed. Among them, for the relationship mining between entities, the present invention determines the spatial relationship between them through the location information such as the longitude and latitude of the entities in the actual implementation process; determines the flow of people transfer relationship between regional entities through massive flow of people data; determines the functional similarity relationship through the distribution of regional POIs; determines the access co-occurrence relationship through user access data and time window control; determines the relevant brands through a preset knowledge base, and determines the relevant brand relationship according to the category to which the brand belongs; determines the location relationship between brand entities and regional entities through existing site selection data; determines other semantic relationships through attribute mining, matching and processing of entities; determines the user's residence relationship, work place relationship and other access relationships through the user's movement trajectory and stay time points; and mines cross-modal relationship facts by matching the geographic information of street view and remote sensing image data with the longitude and latitude boundaries of the region.

[0077] Among them, if we want to build a multimodal urban knowledge graph through the massive multi-source urban spatiotemporal data of city A, the massive multi-source urban spatiotemporal data and statistical information that can be obtained are shown in the following table:

[0078] Table 2 Statistical information of massive multi-source city data

[0079] data Format Data Source quantity Road network data (region id, latitude and longitude boundaries) Map services 500 Business district data (Business district id, name, longitude and latitude) Life service platform 160 POI data (POIid, name, latitude and longitude, category) Map services 22k Brand data (brand id, name) Text Mining 400 Site selection data (brand id, region id) Map services 25k Trajectory data (user id, {latitude and longitude, time} sequence) Mobile Operators 400k Accessing data (userid, POIid) Social Media 1M Street View Data (Street view image, latitude and longitude) Map services 10k Remote sensing data (Remote sensing image, longitude and latitude boundaries) Map services 1k

[0080] The following is a detailed description of the process of constructing a multimodal urban knowledge graph in the present invention. The specific steps are as follows: determine the time span, add the basic time units such as year, month, day, hour, holidays, working days, etc. as entities to the multimodal urban knowledge graph; extract regional entities based on road network data, divide the road network, and determine the regional schematic diagram within the Fifth Ring Road based on the road network. Figure 5 and 6 As shown, Figure 6 The points in the figure are regional entities 601; the business district entities are divided according to expert knowledge. Figure 7 and 8 It shows the central location of all business districts in City A and the service radius of a single business district. Figure 7 The points in are business district entities. Figure 8 The 3km in the figure is the radius of a business district entity; the point of interest entities are matched according to the buildings, stores and infrastructure in the city. Fig. 9The distribution of all points of interest in city A is shown; word segmentation and maximum string matching are performed based on the names of the points of interest to extract brand entities with conceptual significance; user entities in the multimodal city knowledge graph are determined based on the input user data, and each user is an entity with a unique ID; each category is added to the multimodal city knowledge graph as an entity based on the three-level category system divided by expert knowledge; street view images and remote sensing images are matched to the areas divided in the above steps based on the shooting location and range, and a set of street view images corresponding to the boundaries of each area is taken as an entity on the multimodal city knowledge graph, and the remote sensing satellite images corresponding to each area are taken as an entity on the multimodal city knowledge graph; the intersection and proximity relationship definitions between areas are calculated based on the longitude and latitude of the area boundaries, and the corresponding relationship facts are extracted, such as Fig.10 As shown in the figure, it is a schematic diagram of the boundary (top) and adjacent (bottom) relationship facts. The black area is the corresponding head entity, and the others are the corresponding tail entities linked by the relationship. The location relationship fact between the area and the point of interest is calculated based on the longitude and latitude of the area boundary and the longitude and latitude of the point of interest, as shown in Fig.11 As shown in the figure, it is the positional relationship between the area and the point of interest. The outer frame line is the selected area, and each point represents the point of interest located in the area; according to the longitude and latitude of the business district and the longitude and latitude of the area boundary, the service radiation relationship fact between the business district and the area is calculated; according to the longitude and latitude of the point of interest and the longitude and latitude of the business district, the belonging relationship fact between the point of interest and the business district is calculated; according to the longitude and latitude of any two points of interest and the category and brand, the competition relationship fact between the points of interest is calculated, so that two points of interest belonging to the same category and related brands, and the physical distance is less than a certain threshold, constitute a competition relationship on the multimodal urban knowledge graph; according to the name of the point of interest and the brand name, the affiliation relationship fact between the point of interest entity and the brand entity is determined by matching; according to the three-level category system, the parent-child hierarchical relationship fact between the entities of different levels of categories is determined; according to the expert knowledge, the category of the point of interest and the category of the brand are determined, and the attribute relationship fact based on the category is constructed; according to the trajectory data According to the traffic data of different regions, the regional entity pairs and traffic values ​​with traffic transfer at different times are determined by statistics, and the traffic transfer relationship facts are constructed; the similarity between two regions is calculated according to the distribution vector of the points of interest in each region, and the regional entity pairs exceeding the threshold are linked through functional similarity relationship facts; the frequency of continuous visits of two points of interest entities is determined according to the data of user visits to the points of interest, and the points of interest entities with higher frequency, that is, stronger co-occurrence, are linked through visit co-occurrence relationships; brand entity pairs with semantically related relationships are determined according to the relevant pages in the preset encyclopedia knowledge base; the user's work place and residence area are determined according to the stay points, stay time, etc. in the user trajectory data to determine the work place and residence relationship facts, and the high-frequency visit relationship facts and visit relationship facts are determined according to the frequency of user visits to the points of interest; the street view image entity associated with each regional entity is determined according to the regional latitude and longitude boundaries and the shooting location of the street view image, such as Fig.12 The street view entity corresponding to one of the regional entities is shown in FIG. 1 ; the remote sensing image entity associated with each regional entity is determined according to the regional latitude and longitude boundaries and the shooting location of the remote sensing image, such as Fig.12 The figure shows the remote sensing image entity corresponding to one of the regional entities; extracting points of interest and event information based on news events and other data, adding event entities to the multimodal urban knowledge graph, and constructing event relationships; completing the urban resource prediction based on the multimodal urban knowledge graph. Figure 5 , 7 The horizontal and vertical coordinates of , 9, and 11 are longitude and latitude respectively.

[0081] Step 105: Analyze the urban resources of the target object in the target area based on the multimodal urban knowledge graph, and output the analysis results of the urban resources corresponding to the target object in the target area.

[0082] Specifically, first determine the target object to be analyzed (such as brand A) and the corresponding target area (such as the location area of ​​brand A store), use the prediction model based on the multimodal urban knowledge graph to match and analyze the urban resources of the target object in the target area, and predict whether the target area is suitable for opening a brand A store based on the corresponding urban resources, that is, predict whether there is an edge between the target object and the target area in the multimodal urban knowledge graph, and the type of this edge is suitable for opening a store. If suitable, the result that the target area is suitable for opening a brand A store and the urban resource analysis result that the target area is suitable for opening a brand A store are output as the judgment basis. For example, the urban resource analysis result can be that the target area is close to the railway station or close to a certain business district, so it is suitable for opening a brand A store, thereby achieving the characterization of the relationship combination between the target object and the area on the multimodal urban knowledge graph. Among them, the multimodal urban knowledge graph pre-contains the urban resource information corresponding to the brand A store in other areas. The brand A can be the brand id of a catering brand or the brand id of a clothing brand, which is not specifically limited here.

[0083] The urban resource prediction method based on the multimodal urban knowledge graph described in the embodiment of the present invention distills knowledge from massive multi-source urban spatiotemporal data, first abstracts an ontology set with urban basic elements as the core, then matches specific entity objects from the multi-source urban spatiotemporal data, further defines the corresponding association relationship set by mining the semantic association relationship between the ontologies, and finally performs knowledge fusion based on the matched entity objects, the constructed association relationship and the multi-source urban spatiotemporal data to complete the construction of the multi-source urban knowledge graph. It can quickly match and fuse the multi-source urban spatiotemporal data into the ontology and entities of the urban knowledge graph structure, and mine the semantic relationship of the urban spatiotemporal data, thereby improving the efficiency and accuracy of urban resource prediction based on the multimodal urban knowledge graph.

[0084] Corresponding to the above-mentioned method for predicting urban resources based on a multimodal urban knowledge graph, the present invention also provides an urban resource prediction system based on a multimodal urban knowledge graph. Since the embodiment of the system is similar to the above-mentioned method embodiment, the description is relatively simple. For relevant details, please refer to the description of the above-mentioned method embodiment. The embodiment of the urban resource prediction system based on a multimodal urban knowledge graph described below is only illustrative. Please refer to Fig.13 As shown, it is a structural diagram of an urban resource prediction system based on a multimodal urban knowledge graph provided by an embodiment of the present invention.

[0085] The urban resource prediction system based on multimodal urban knowledge graph described in the present invention specifically includes the following parts:

[0086] The ontology determination unit 1301 is used to obtain multi-source urban spatiotemporal data, perform ontology abstraction based on the multi-source urban spatiotemporal data, and determine an ontology set corresponding to basic elements of the city;

[0087] An entity object determination unit 1302 is used to perform entity matching based on the multi-source urban spatiotemporal data and the ontology set to determine an entity object set corresponding to the ontology;

[0088] An association relationship determination unit 1303 is used to construct relationships based on the multi-source urban spatiotemporal data and the ontology set, and determine an association relationship set between ontologies;

[0089] The city knowledge graph determination unit 1304 is used to perform knowledge fusion based on the entity object set, the association relationship set and the multi-source city spatiotemporal data to obtain a corresponding multimodal city knowledge graph.

[0090] Before performing ontology abstraction based on the multi-source urban spatiotemporal data and determining the ontology set corresponding to the basic elements of the city, the method further includes: determining the structure of the urban knowledge graph based on the graph pattern, and determining the types of entities in the urban knowledge graph and the relationships between various ontologies;

[0091] The ontology determination unit is specifically used to: perform ontology abstraction from the multi-source urban spatiotemporal data based on the type of the entity and the relationship between the various ontologies to obtain an ontology set corresponding to the basic elements of the city; the ontology set includes a thing ontology, a space ontology and a time ontology; wherein the space ontology includes a region ontology, a business district ontology and a point of interest ontology; the thing ontology includes a user ontology, a brand ontology and corresponding attribute ontology; the time ontology includes timestamp information of different scales and time concept information corresponding to specific semantics; the attribute ontology corresponds to the attribute information of each ontology.

[0092] Furthermore, the entity object determination unit is specifically used to:

[0093] Perform entity matching based on the multi-source urban spatiotemporal data and the ontology set to determine an entity set; perform attribute enhancement based on the entity set to determine corresponding entity attributes; obtain the entity object set based on the entity set and the entity attributes;

[0094] Among them, the entity object set includes the entity set and the entity attributes; the entity set includes the thing entity corresponding to the thing entity, the space entity corresponding to the space entity and the time entity corresponding to the time entity; the entity attributes correspond to the attribute entity, and the entity attributes include holiday information, location area information, activity level information, user portrait information, category brand information, street view remote sensing information and user portrait information.

[0095] Furthermore, the association relationship determination unit is specifically used to: construct relationships based on the multi-source urban spatiotemporal data and the ontology set to determine the physical level relationships of the ontology; and perform relationship enhancement processing based on the physical level relationships to determine the semantic level relationships of the ontology; wherein the association relationship set includes the physical level relationships and the semantic level relationships; the physical level relationships include the positional relationships, temporal relationships and sequence relationships of the ontology; and the semantic level relationships include the attribute relationships, functional relationships, transfer relationships, competitive relationships, causal relationships and co-occurrence relationships of the ontology.

[0096] Furthermore, the city knowledge graph determination unit is specifically used to:

[0097] Abstracting the multi-source urban spatiotemporal data into several layers, and determining the ontology corresponding to each layer;

[0098] The multi-source urban spatiotemporal data are analyzed for association relationships, with the relationship mining between entities and the relationship mining between ontologies being carried out, and the instantiation matching process from ontology to entity is realized, knowledge fusion is performed, the corresponding relationship facts are extracted, and the construction of the urban knowledge graph is completed to obtain the corresponding multimodal urban knowledge graph.

[0099] Furthermore, the relationship mining between entities includes:

[0100] Determine the spatial relationship between entities based on their longitude and latitude location information; determine the flow of people transfer relationship between regional entities through massive flow of people data; determine the functional similarity relationship based on the distribution of points of interest in the region; determine the visit co-occurrence relationship based on user access data and time window control; determine the relevant brand information based on the preset Chinese knowledge base, and determine the corresponding category brand relationship according to the category to which the brand belongs; determine the location relationship between brand entities and regional entities based on existing site selection data; determine the corresponding semantic relationship based on the attribute mining, matching and processing of entities; determine the user's residence relationship, work place relationship and corresponding visit relationship based on the user's movement trajectory and stay time point; match the geographic information of street view and remote sensing image data with the longitude and latitude boundaries of the region to mine cross-modal relationship facts.

[0101] The urban resource prediction system based on the multimodal urban knowledge graph described in the embodiment of the present invention distills knowledge from massive multi-source urban spatiotemporal data, first abstracts an ontology set with urban basic elements as the core, then matches specific entity objects from the multi-source urban spatiotemporal data, further defines the corresponding association relationship set by mining the semantic association relationship between the ontologies, and finally performs knowledge fusion based on the matched entity objects, constructed association relationships and multi-source urban spatiotemporal data to complete the construction of the multi-source urban knowledge graph. It can quickly match and fuse the multi-source urban spatiotemporal data into the ontology and entities of the urban knowledge graph structure, and mine the semantic relationships of the urban spatiotemporal data, thereby improving the efficiency and accuracy of urban resource prediction based on the multimodal urban knowledge graph.

[0102] Corresponding to the above-mentioned urban resource prediction method based on multimodal urban knowledge graph, the present invention also provides an electronic device. Since the embodiment of the electronic device is similar to the above-mentioned method embodiment, the description is relatively simple. For relevant parts, please refer to the description of the above-mentioned method embodiment. The electronic device described below is only exemplary. Fig.14As shown, it is a schematic diagram of the entity structure of an electronic device disclosed in an embodiment of the present invention. The electronic device may include: a processor 1401, a memory 1402, a communication bus 1403 (i.e., the above-mentioned system bus) and a search engine 1405, wherein the processor 1401 and the memory 1402 complete mutual communication through the communication bus 1403, and communicate with the outside through the communication interface 1404. The processor 1401 can call the logic instructions in the memory 1402 to execute the urban resource prediction method based on the multimodal urban knowledge graph, the method comprising: obtaining multi-source urban spatiotemporal data, performing ontology abstraction based on the multi-source urban spatiotemporal data, and determining an ontology set corresponding to the basic elements of the city; performing entity matching based on the multi-source urban spatiotemporal data and the ontology set, and determining an entity object set corresponding to the ontology; performing relationship construction based on the multi-source urban spatiotemporal data and the ontology set, and determining an association relationship set between the ontology; performing knowledge fusion based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data, and obtaining a corresponding multimodal urban knowledge graph.

[0103] In addition, the logic instructions in the above-mentioned memory 1402 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: a storage chip, a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, and other media that can store program codes.

[0104] On the other hand, an embodiment of the present invention further provides a computer program product, the computer program product includes a computer program stored on a processor-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer, the computer can execute the urban resource prediction method based on the multimodal urban knowledge graph provided by the above-mentioned method embodiments. The method includes: obtaining multi-source urban spatiotemporal data, performing ontology abstraction based on the multi-source urban spatiotemporal data, and determining an ontology set corresponding to basic elements of the city; performing entity matching based on the multi-source urban spatiotemporal data and the ontology set, and determining an entity object set corresponding to the ontology; performing relationship construction based on the multi-source urban spatiotemporal data and the ontology set, and determining an association relationship set between ontologies; performing knowledge fusion based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data, and obtaining a corresponding multimodal urban knowledge graph.

[0105] On the other hand, an embodiment of the present invention further provides a processor-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor, it is implemented to execute the urban resource prediction method based on the multimodal urban knowledge graph provided in the above embodiments. The method includes: obtaining multi-source urban spatiotemporal data, performing ontology abstraction based on the multi-source urban spatiotemporal data, and determining an ontology set corresponding to basic elements of the city; performing entity matching based on the multi-source urban spatiotemporal data and the ontology set, and determining an entity object set corresponding to the ontology; performing relationship construction based on the multi-source urban spatiotemporal data and the ontology set, and determining an association relationship set between ontologies; performing knowledge fusion based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data, and obtaining a corresponding multimodal urban knowledge graph.

[0106] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0107] The system embodiment described above is merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0108] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0109] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for predicting urban resources based on a multimodal urban knowledge graph, characterized in that: include: Acquire multi-source urban spatiotemporal data, perform ontology abstraction based on the multi-source urban spatiotemporal data, and determine an ontology set corresponding to basic elements of the city; Perform entity matching based on the multi-source urban spatiotemporal data and the ontology set to determine a set of entity objects corresponding to the ontology; Building relationships based on the multi-source urban spatiotemporal data and the ontology set, and determining a set of association relationships between ontologies; The said constructing relationships based on the multi-source urban spatiotemporal data and the ontology set to determine the association relationship set between the ontologies specifically includes: constructing relationships based on the multi-source urban spatiotemporal data and the ontology set to determine the physical level relationships of the ontologies; and performing relationship enhancement processing based on the physical level relationships to determine the semantic level relationships of the ontologies; wherein the association relationship set includes the physical level relationships and the semantic level relationships; the physical level relationships include the positional relationships, temporal relationships, and positional relationships of the ontologies; and the semantic level relationships include the attribute relationships, functional relationships, transfer relationships, competitive relationships, causal relationships, and co-occurrence relationships of the ontologies; Based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data, knowledge fusion is performed to obtain a corresponding multimodal urban knowledge graph; the knowledge fusion based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data to obtain a corresponding multimodal urban knowledge graph specifically includes: abstracting the multi-source urban spatiotemporal data into several layers, and determining the ontology corresponding to each layer; performing association relationship analysis on the multi-source urban spatiotemporal data, mining the relationships between entities and between ontologies, and realizing the instantiation matching process from ontology to entity, performing knowledge fusion, extracting corresponding relationship facts, and completing the construction of the urban knowledge graph to obtain the corresponding multimodal urban knowledge graph; wherein, for the relationship between entities The spatial relationship between entities is determined through the longitude and latitude of entities; the flow of people transfer relationship between regional entities is determined through massive flow of people data; the functional similarity relationship is determined through the distribution of regional POIs; the access co-occurrence relationship is determined through user access data and time window control; the relevant brands are determined through the preset knowledge base, and the relevant brand relationship is determined according to the brand category; the location relationship between brand entities and regional entities is determined through the existing location data; other semantic relationships are determined through the attribute mining, matching and processing of entities; the user's residence relationship, work relationship and other access relationships are determined through the user's movement trajectory and stay time points; the geographical information of street view and remote sensing image data is matched with the regional longitude and latitude boundaries to mine cross-modal relationship facts; Based on the multimodal urban knowledge graph, the urban resources of the target object in the target area are analyzed, and the urban resource analysis results corresponding to the target object in the target area are output.

2. The urban resource prediction method based on multimodal urban knowledge graph according to claim 1 is characterized in that: Before performing ontology abstraction based on the multi-source urban spatiotemporal data to determine an ontology set corresponding to basic elements of the city, the method further includes: Determine the structure of the urban knowledge graph based on the graph model, and determine the types of entities in the urban knowledge graph and the relationships between various ontologies; The ontology abstraction based on the multi-source urban spatiotemporal data to determine the ontology set corresponding to the basic elements of the city specifically includes: performing ontology abstraction from the multi-source urban spatiotemporal data based on the type of the entity and the relationship between the various ontologies to obtain the ontology set corresponding to the basic elements of the city; the ontology set includes a thing ontology, a space ontology and a time ontology; wherein the space ontology includes a region ontology, a business district ontology and a point of interest ontology; the thing ontology includes a user ontology, a brand ontology and corresponding attribute ontology; the time ontology includes timestamp information of different scales and time concept information corresponding to specific semantics; and the attribute ontology corresponds to the attribute information of each ontology.

3. The urban resource prediction method based on multimodal urban knowledge graph according to claim 2 is characterized in that: Performing entity matching based on the multi-source urban spatiotemporal data and the ontology set to determine an entity object set corresponding to the ontology specifically includes: Entity matching is performed based on the multi-source urban spatiotemporal data and the ontology set to determine an entity set; and attribute enhancement is performed based on the entity set to determine corresponding entity attributes; the entity object set is obtained based on the entity set and the entity attributes; wherein the entity object set includes the entity set and the entity attributes; the entity set includes thing entities corresponding to the thing ontology, space entities corresponding to the space ontology, and time entities corresponding to the time ontology; the entity attributes correspond to the attribute ontology, and the entity attributes include holiday information, location area information, activity level information, user portrait information, category brand information, street view remote sensing information and user portrait information.

4. The urban resource prediction method based on multimodal urban knowledge graph according to claim 1 is characterized in that: The relationship mining between entities includes: Determine the spatial relationship between entities based on their longitude and latitude location information; determine the flow of people transfer relationship between regional entities through massive flow of people data; determine the functional similarity relationship based on the distribution of points of interest in the region; determine the visit co-occurrence relationship based on user access data and time window control; determine the relevant brand information based on the preset Chinese knowledge base, and determine the corresponding category brand relationship according to the category to which the brand belongs; determine the location relationship between brand entities and regional entities based on existing site selection data; determine the corresponding semantic relationship based on the attribute mining, matching and processing of entities; determine the user's residence relationship, work place relationship and corresponding visit relationship based on the user's movement trajectory and stay time point; match the geographic information of street view and remote sensing image data with the longitude and latitude boundaries of the region to mine cross-modal relationship facts.

5. An urban resource prediction system based on a multimodal urban knowledge graph, characterized in that: include: An ontology determination unit, used to obtain multi-source urban spatiotemporal data, perform ontology abstraction based on the multi-source urban spatiotemporal data, and determine an ontology set corresponding to basic elements of the city; An entity object determination unit, configured to perform entity matching based on the multi-source urban spatiotemporal data and the ontology set, and determine an entity object set corresponding to the ontology; The said constructing relationships based on the multi-source urban spatiotemporal data and the ontology set to determine the association relationship set between the ontologies specifically includes: constructing relationships based on the multi-source urban spatiotemporal data and the ontology set to determine the physical level relationships of the ontologies; and performing relationship enhancement processing based on the physical level relationships to determine the semantic level relationships of the ontologies; wherein the association relationship set includes the physical level relationships and the semantic level relationships; the physical level relationships include the positional relationships, temporal relationships, and positional relationships of the ontologies; and the semantic level relationships include the attribute relationships, functional relationships, transfer relationships, competitive relationships, causal relationships, and co-occurrence relationships of the ontologies; An association relationship determination unit, configured to construct relationships based on the multi-source urban spatiotemporal data and the ontology set, and determine an association relationship set between ontologies; The urban knowledge graph determination unit is used to perform knowledge fusion based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data to obtain a corresponding multimodal urban knowledge graph; the knowledge fusion based on the entity object set, the association relationship set and the multi-source urban spatiotemporal data to obtain a corresponding multimodal urban knowledge graph specifically includes: abstracting the multi-source urban spatiotemporal data into several layers, and determining the ontology corresponding to each layer; performing association relationship analysis on the multi-source urban spatiotemporal data, mining the relationships between entities and between ontologies, and realizing the instantiation matching process from ontology to entity, performing knowledge fusion, extracting corresponding relationship facts, and completing the construction of the urban knowledge graph to obtain the corresponding multimodal urban knowledge graph; wherein , for the relationship mining between entities, the spatial relationship between them is determined through the longitude and latitude of the entities; the flow of people transfer relationship between regional entities is determined through massive flow of people data; the functional similarity relationship is determined through the distribution of regional POIs; the access co-occurrence relationship is determined through user access data and time window control; the relevant brands are determined through the preset knowledge base, and the relevant brand relationship is determined according to the category to which the brand belongs; the location relationship between brand entities and regional entities is determined through the existing location data; other semantic relationships are determined by mining, matching and processing the attributes of entities; the user's residence relationship, work relationship and other access relationships are determined through the user's movement trajectory and stay time points; the geographical information of street view and remote sensing image data is matched with the longitude and latitude boundaries of the region to mine cross-modal relationship facts; The urban resource analysis unit is used to analyze the urban resources of the target object in the target area based on the multimodal urban knowledge graph, and output the urban resource analysis results corresponding to the target object in the target area.

6. The urban resource prediction system based on multimodal urban knowledge graph according to claim 5 is characterized in that: The city knowledge graph determination unit is specifically used to: Abstracting the multi-source urban spatiotemporal data into several layers, and determining the ontology corresponding to each layer; The multi-source urban spatiotemporal data are analyzed for association relationships, with the relationship mining between entities and the relationship mining between ontologies being carried out, and the instantiation matching process from ontology to entity is realized, knowledge fusion is performed, corresponding relationship facts are extracted, and the construction of the urban knowledge graph is completed to obtain the corresponding multimodal urban knowledge graph.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the urban resource prediction method based on the multimodal urban knowledge graph as described in any one of claims 1 to 4 are implemented.

8. A processor-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the urban resource prediction method based on a multimodal urban knowledge graph as described in any one of claims 1 to 4 are implemented.

Images