Spatio-temporal data analysis method and graph calculation system supporting spatio-temporal data analysis

Abstract

The invention discloses a spatio-temporal data analysis method and a graph calculation system supporting spatio-temporal data analysis. The method comprises the following steps: performing parallel execution in each alternative node in the graph computing system: generating a plurality of target graph data corresponding to a plurality of original spatio-temporal data according to a preset graph data structure and the plurality of original spatio-temporal data in the alternative nodes; according to a target partitioning mode, a plurality of target graph data in the alternative nodes are distributed to target nodes matched with the target graph data, and the graph data structure and the target partitioning mode are determined according to analysis demand information of the spatio-temporal data; calling a target analysis algorithm in a pre-constructed algorithm library according to the target graph data and the analysis demand information, and executing the target analysis algorithm on the target graph data to obtain an analysis result. According to the embodiment of the invention, one set of computing system is compatible with spatio-temporal data and graph data analysis at the same time, and construction of multiple sets of computing systems is avoided.

Description

Technical Field

[0001] The present application belongs to the field of big data processing technology, and in particular, relates to a spatiotemporal data analysis method and a graph computing system supporting spatiotemporal data analysis. Background Art

[0002] The algorithms and methods of spatiotemporal data analysis have many similarities with graph algorithms, such as cluster analysis and path analysis, which are the main analysis methods for spatiotemporal data and graph data. However, in terms of computing systems, the two rely on different computing systems. For users who need to process spatiotemporal data and graph data at the same time, it will cause a huge waste of resources. Summary of the invention

[0003] The embodiments of the present application provide a spatiotemporal data analysis method and a graph computing system supporting spatiotemporal data analysis, so as to at least solve the problem in the related art that the graph computing system does not support spatiotemporal data processing.

[0004] In a first aspect, an embodiment of the present application provides a spatiotemporal data analysis method, the method being applied to a graph computing system, the graph computing system being a distributed computing system;

[0005] The method comprises:

[0006] The following steps A and B are performed in parallel in each candidate node in the graph computing system:

[0007] Step A: generating a plurality of target graph data corresponding to the plurality of original spatiotemporal data respectively according to a preset graph data structure and a plurality of original spatiotemporal data in the candidate nodes, wherein the graph data structure is determined according to the analysis requirement information of the spatiotemporal data;

[0008] Step B: according to the target partitioning method, respectively allocating a number of target graph data in the candidate nodes to target nodes matching the target graph data, wherein the target partitioning method is determined according to the analysis requirement information;

[0009] The following steps C are executed in parallel on each target node:

[0010] Step C: calling a target analysis algorithm in a pre-built algorithm library according to the target graph data and analysis requirement information, and executing the target analysis algorithm on the target graph data to obtain an analysis result.

[0011] In a second aspect, an embodiment of the present application provides a graph computing system that supports spatiotemporal data analysis, wherein the graph computing system is a distributed computing system and includes multiple candidate nodes;

[0012] The graph computing system includes a data loader, a data partitioner and distributor, a memory storage, and a computing engine;

[0013] The data loader is used to input the original spatiotemporal data into the graph computing system, and generate a plurality of target graph data corresponding to the plurality of original spatiotemporal data according to a preset graph data structure and a plurality of original spatiotemporal data in the candidate nodes, wherein the graph data structure is determined according to the analysis requirement information of the spatiotemporal data;

[0014] The data partitioner and distributor is used to distribute the target graph data in the candidate nodes to the target nodes matching the target graph data according to the target partitioning method, wherein the target partitioning method is determined according to the analysis requirement information, and store the target graph data in the memory storage of the corresponding target nodes respectively;

[0015] The computing engine is used to read the target graph data in the memory storage, call the target analysis algorithm in the pre-built algorithm library according to the target graph data and analysis requirement information, and execute the target analysis algorithm on the target graph data to obtain the analysis result.

[0016] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the steps of the spatiotemporal data analysis method described in any one of the embodiments of the first aspect are implemented.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the steps of the spatiotemporal data analysis method as described in any one of the embodiments of the first aspect are implemented.

[0018] In a fifth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the steps of the spatiotemporal data analysis method provided in the first aspect of the embodiment of the present application.

[0019] The spatiotemporal data analysis method and the graph computing system supporting spatiotemporal data analysis of the embodiments of the present application reuse the distributed computing and communication framework of the graph computing system, load the original spatiotemporal data into the graph computing system, and convert the input spatiotemporal data into converted graph data according to a preset graph data structure; repartition and distribute the converted graph data according to the target partitioning method to place adjacent data in the same partition, thereby reducing frequent network transmission during the calculation process and improving computing performance; execute the analysis algorithm specified by the computing task on the graph data formed after the conversion and distribution of the spatiotemporal data to obtain the analysis result, so as to meet the analysis requirements of the spatiotemporal data, and realize a computing system that is compatible with both spatiotemporal data and graph data analysis, thus avoiding the construction of multiple computing systems and saving software and hardware costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solution of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 It is a flow chart of a spatiotemporal data analysis method provided in an embodiment of the present application;

[0022] Figure 2 is a schematic diagram of an exemplary graph data structure provided in an embodiment of the present application;

[0023] Figure 3 It is a flowchart of an exemplary method for constructing a target analysis algorithm provided in an embodiment of the present application;

[0024] Figure 4 It is a flowchart of another spatiotemporal data analysis method provided in an embodiment of the present application;

[0025] Figure 5 It is a structural schematic diagram of a graph computing system supporting spatiotemporal data analysis provided by an embodiment of the present application;

[0026] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0027] Reference numerals:

[0028] Graph computing system 500 supporting spatiotemporal data analysis, data loader 501, data partition and distributor 502, memory storage 503, computing engine 504,

[0029] Electronic device 600 , processor 601 , memory 602 , communication interface 603 , bus 610 . DETAILED DESCRIPTION

[0030] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0031] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0032] Spatiotemporal data generally refers to data that contains both temporal and spatial information and comes from systems such as geographic information systems, sensor networks, social media, and mobile applications.

[0033] Spatiotemporal data analysis refers to pattern recognition, trend analysis, anomaly detection, clustering, path analysis, etc. of spatiotemporal data to explore and gain insights into the hidden value in the data. Currently, the main means of spatiotemporal analysis are through spatiotemporal databases, such as PostGIS, Oracle Spatial, etc. There are also big data technologies (such as Hadoop, Spark) that process and analyze spatiotemporal data.

[0034] The graph computing system is a system specially designed for processing and analyzing graph data. It realizes efficient operation and analysis of complex graph data through optimized storage architecture, query language, graph processing framework and graph algorithm library. The graph computing system can be used as a module of the graph database (such as Neo4J) or as a separate computing system (such as GraphX, GraphScope, Plato, etc.).

[0035] The algorithms and methods of spatiotemporal data analysis have many similarities with graph algorithms, such as cluster analysis and path analysis, which are the main analysis methods for spatiotemporal data and graph data. However, in terms of computing systems, the two rely on different computing systems. For users who need to process spatiotemporal data and graph data at the same time, it will cause a huge waste of resources, such as but not limited to: at the software system level, it is necessary to purchase and maintain two computing systems; at the hardware resource level, it is necessary to provide the resources required by the two computing systems; at the algorithm level, it is necessary to develop and maintain spatiotemporal analysis algorithms and graph analysis algorithms.

[0036] Currently, there are not many mature methods for processing spatiotemporal data using graph technology. The more common one is Neo4j's support for spatiotemporal data, which provides plug-ins for storing and querying spatiotemporal data. Simple analysis of spatiotemporal data can be performed through query. However, Neo4j essentially solves the problem of co-storage and query of graph data and spatiotemporal data. It only supports the storage and query of spatiotemporal data, and does not support running complex algorithms on spatiotemporal data, such as path analysis algorithms, clustering algorithms, subgraph pattern matching, etc.

[0037] In order to solve the problems of related technologies, an embodiment of the present application provides a spatiotemporal data analysis method and a graph computing system supporting spatiotemporal data analysis.

[0038] The spatiotemporal data analysis method provided in the embodiment of the present application is described in detail below through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0039] It should be noted that the spatiotemporal data analysis method of the embodiment of the present application is applied to a graph computing system, and the graph computing system is a distributed computing system, that is, the distributed computing and communication framework of the graph computing system can be reused.

[0040] In the embodiment of the present application, all devices and components in the graph computing system retain the existing ability to process graph data. Therefore, in all the following descriptions, how to process graph data compatibly is no longer described in detail, but only how to process spatiotemporal data is explained in detail.

[0041] Figure 1 FIG. 1 is a flow chart of a spatiotemporal data analysis method 100 according to an embodiment of the present application. Figure 1 As shown, the spatiotemporal data analysis method 100 may specifically include the following steps:

[0042] S101, performing the following steps A and B in parallel in each candidate node in the graph computing system:

[0043] Step A: generating a plurality of target graph data corresponding to the plurality of original spatiotemporal data respectively according to a preset graph data structure and a plurality of original spatiotemporal data in the candidate nodes, wherein the graph data structure is determined according to the analysis requirement information of the spatiotemporal data;

[0044] Step B: according to the target partitioning method, respectively allocating a number of target graph data in the candidate nodes to target nodes matching the target graph data, wherein the target partitioning method is determined according to the analysis requirement information;

[0045] S102, executing the following steps C in parallel in each target node:

[0046] Step C: calling a target analysis algorithm in a pre-built algorithm library according to the target graph data and analysis requirement information, and executing the target analysis algorithm on the target graph data to obtain an analysis result.

[0047] Therefore, the distributed computing and communication framework of the graph computing system is reused, the original spatiotemporal data is loaded into the graph computing system, and the input spatiotemporal data is converted into graph data according to the preset graph data structure; the converted graph data is repartitioned and allocated according to the target partitioning method to put adjacent data into the same partition, reducing frequent network transmission during the calculation process and improving computing performance; the analysis algorithm specified by the computing task is executed on the graph data formed after the conversion and allocation of the spatiotemporal data to obtain the analysis results, which meets the analysis requirements of the spatiotemporal data and realizes a computing system that is compatible with both spatiotemporal data and graph data analysis, avoiding the construction of multiple computing systems and saving software and hardware costs.

[0048] The specific implementation methods of the above steps are introduced below.

[0049] It should be noted that in the embodiments of the present application, in order to efficiently load data and improve data processing efficiency, each computing node in a distributed mode in the graph computing system loads and processes data in parallel.

[0050] In some embodiments, before S101, the graph data structure Schema is designed according to the analysis requirement information of the predetermined spatiotemporal data. Specifically, a suitable design can be made according to the actual application requirements and the purpose of the spatiotemporal analysis, and the node types, edge types and their specific meanings of the graph data structure can be predefined. In addition, it can be understood that the original spatiotemporal data may vary depending on the acquisition device or data source. Therefore, in S101, the corresponding graph data structure Schema needs to refer to the specific information contained in the original spatiotemporal data.

[0051] Specifically, the format of the original spatiotemporal data is a comma-separated value CSV file. Take the original spatiotemporal data including Locations.csv, Events.csv, Sensors.csv, and Relations.csv as an example:

[0052] Locations.csv

[0053] id,name,latitude,longitude

[0054] 1,Central Park,40.785091,-73.968285

[0055] 2,Times Square,40.758896,-73.985130

[0056] 3,Brooklyn Bridge,40.706086,-73.996864

[0057] 4,Empire State Building,40.748817,-73.985428

[0058] 5,Statue ofLiberty,40.689247,-74.044502

[0059] Events.csv

[0060] id,name,timestamp

[0061] 1,Concert,2024-01-01T20:00:00Z

[0062] 2,New Year Celebration,2024-01-01T00:00:00Z

[0063] 3,Parade,2024-01-01T10:00:00Z

[0064] 4,Fireworks,2024-01-01T21:00:00Z

[0065] 5,Marathon,2024-01-01T08:00:00Z

[0066] 6,Food Festival,2024-01-02T12:00:00Z

[0067] 7,Art Exhibition,2024-01-02T14:00:00Z

[0068] 8,Tech Conference,2024-01-02T09:00:00Z

[0069] 9,Book Fair,2024-01-02T11:00:00Z

[0070] 10,Music Festival,2024-01-02T17:00:00Z11,Charity Run,2024-01-03T07:00:00Z12,Theater Performance,2024-01-03T19:00:00Z13,Science Fair,2024-01-03T10:00:00Z14,Film Premiere,2024-01-03T20:00:00Z15,Community Meetup,2024-01-03T18:00:00Z Sensors.csv

[0071] id,name,latitude,longitude1,Sensor A,40.785091,-73.968285

[0072] 2,Sensor B,40.758896,-73.985130

[0073] 3,Sensor C,40.706086,-73.996864

[0074] 4,Sensor D,40.748817,-73.985428

[0075] 5,Sensor E,40.689247,-74.044502

[0076] Relations.csv

[0077] event_id,location_id,sensor_id

[0078] 1,1,1

[0079] 2,2,2

[0080] 3,1,3

[0081] 4,2,4

[0082] 5,3,5

[0083] 6,3,1

[0084] 7,4,2

[0085] 8,5,3

[0086] 9,1,4

[0087] 10,2,5

[0088] 11,3,1

[0089] 12,4,2

[0090] 13,5,3

[0091] 14,1,4

[0092] 15,2,5

[0093] It can be understood that in order to import these four types of raw spatiotemporal data into the graph computing system, it is necessary to convert them into corresponding target graph data. Then, according to the raw spatiotemporal data and analysis requirement information, an exemplary graph data structure can be designed, referring to Figure 2 .like Figure 2 As shown, the graph data structure includes node types and edge types.

[0094] The node types include:

[0095] Location: indicates the geographical location, including attributes such as latitude and longitude;

[0096] Event: represents an event, including timestamp and other related attributes;

[0097] Sensor: represents the sensor, including attributes such as location and time.

[0098] Edge types include:

[0099] OCCURRED_AT: indicates that the event occurred at a certain location;

[0100] RECORDED_BY: indicates that the event was recorded by a sensor;

[0101] NEXT: Indicates the sequential relationship in time and connects events that are adjacent in time.

[0102] It should be noted that the above graph data structure schema cannot summarize the conversion of all spatiotemporal data to graph data. The original schema of spatiotemporal data may also vary depending on the acquisition device or data source. The corresponding graph data schema needs to refer to the original schema of the spatiotemporal data and the purpose of spatiotemporal analysis to make an appropriate design.

[0103] For example, if the original spatiotemporal data collected by the sensor includes not only Location, ID, and Sensor data, but also information such as humidity and temperature, and it can be understood that which data in the original spatiotemporal data needs to be analyzed and processed is determined according to the analysis requirements and purposes. Then the graph data structure can be designed accordingly.

[0104] Therefore, based on the above graph data structure and the original spatiotemporal data, by converting the csv file of the spatiotemporal data into graph data, the graph computing system can support the import of spatiotemporal data types.

[0105] Furthermore, in some embodiments, a target partitioning method matching the analysis requirement information is determined from among several preset candidate partitioning methods. Specifically, the candidate partitioning methods include: hash partitioning, range partitioning, space partitioning, and time partitioning.

[0106] Among them, spatial partitioning includes: determining the number of candidate nodes in the graph computing system as the number of clusters, spatially clustering the data, obtaining the target cluster number corresponding to the data, and the cluster number corresponds to the candidate node one by one; querying the matching target node from several candidate nodes according to the target cluster number.

[0107] Time partitioning includes: determining the number of candidate nodes in the graph computing system as the number of clusters, performing time clustering on the data, obtaining the target cluster number corresponding to the data, and the cluster number corresponds to the candidate node one by one; and querying the matching target node from several candidate nodes according to the target cluster number.

[0108] In this way, in addition to general hash partitioning and range partitioning, the partitioning methods are supplemented by designing space partitioning and time partitioning methods, which can fully consider the characteristics of spatiotemporal data when partitioning and make the localized distribution of data more suitable for spatiotemporal analysis logic.

[0109] In addition, in some embodiments, a spatiotemporal analysis algorithm is constructed and stored in an algorithm library. It should be noted that a graph computing system usually has a rich built-in general graph algorithm library, such as path algorithms, clustering algorithms, node importance algorithms, etc. The spatiotemporal data has been converted into graph data storage in the graph computing system, and naturally these graph algorithms can be seamlessly used for graph analysis. Therefore, for the spatiotemporal analysis algorithms corresponding to spatiotemporal analysis, some general spatiotemporal algorithms can be generated by simple extension of graph algorithms, and customized algorithms can be developed through the algorithm development interface of the graph computing system.

[0110] In specific implementation, the custom algorithm interface of the graph computing system can be used to implement customized computing logic. Since the spatiotemporal data has been converted into graph data, and the related data fields such as space and time have been converted into the attributes of the points or edges of the graph data, the spatiotemporal data can be obtained by reading the attributes in the graph data through the custom algorithm interface, and then the computing logic can be processed.

[0111] In the specific implementation, it can be expanded based on the existing graph algorithm. It includes: splitting the alternative analysis algorithms in the algorithm library to obtain the general algorithm logic compatible with graph analysis and spatiotemporal analysis and the graph algorithm logic applied to graph analysis; determining the general algorithm logic as the target analysis base class algorithm, and determining the graph algorithm logic as the target analysis subclass graph algorithm; creating the target analysis subclass spatiotemporal algorithm according to the target analysis base class algorithm; determining the target analysis base class algorithm, the target analysis subclass graph algorithm and the target analysis subclass spatiotemporal algorithm as the target analysis algorithm, and storing the target analysis algorithm in the algorithm library.

[0112] In other words, the built-in graph algorithm code of the graph computing system can be optimized based on object-oriented thinking. For the entire set of algorithm logic, the "template method" design pattern is adopted. Most of the common algorithm logic is placed in the base class. When some links have different logics for processing spatiotemporal data and graph data, they are implemented in the spatiotemporal algorithm subclass and the graph algorithm subclass respectively. In this way, most of the existing graph algorithm logic is reused, and a small amount of logic is expanded to achieve the purpose of spatiotemporal analysis.

[0113] refer to Figure 3 , is an exemplary method for constructing a target analysis algorithm. Figure 3 As shown in the figure, taking the PathAnalysis algorithm as an example, the implementation of getDistance in the algorithm is different in graph analysis and spatiotemporal analysis, while the other parts of the algorithm are basically the same. Therefore, the algorithm can be split into the BasePathAnalysis base class and the GraphPathAnalysis subclass. Most of the calculation logic is implemented in the base class, and the getDistance method is set as a pure virtual function. The getDistance function is implemented in the GraphPathAnalysis subclass to calculate the distance between two points on the graph. For spatiotemporal analysis, you only need to create a TSPathAnalysis subclass based on BasePathAnalysis and implement the getDistance method to calculate the distance logic between spatiotemporal data to obtain the path analysis algorithm for spatiotemporal analysis. It should be understood that in the two subclasses, only getDistance is implemented for different data types, and the other methods in the two subclasses are inherited from its parent class and do not need to be rewritten. In this way, for data processing of different data types, the corresponding subclass algorithm can be directly called for calculation.

[0114] Thus, based on the constructed algorithm library, the target analysis algorithm in the pre-constructed algorithm library is called according to the target graph data and the analysis requirement information, and the target analysis algorithm is executed on the target graph data to obtain the analysis result, that is, S102 is executed.

[0115] In addition, reference Figure 4, an embodiment of the present application provides another spatiotemporal data analysis method.

[0116] like Figure 4 As shown, in some embodiments, the data loader converts and loads external spatiotemporal data into the graph computing system according to the definition of "graph data schema". The graph data schema is the schema of the graph data corresponding to the spatiotemporal data that is about to enter the graph computing system, and the data loader uses this as the target graph data structure to convert the input spatiotemporal data.

[0117] In some embodiments, in a distributed mode, after each node loads the spatiotemporal data in parallel, the data is partitioned according to the set partitioning strategy through the data partitioner and distributor, and the data is redistributed in each node through the shuffle process so that the data is stored in the memory storage of each node according to the partitioning strategy. In this way, adjacent data can be placed in the same partition, reducing frequent network transmission during the calculation process and improving the calculation performance.

[0118] It should be noted that multiple strategies are defined. When doing spatiotemporal analysis, you need to choose an appropriate partitioning strategy based on the purpose of the analysis and the characteristics of the data.

[0119] Optionally, the partitioning strategy includes a spatial strategy. The location information of the spatiotemporal data is entered into the graph and stored as a point type of Location. In order to reasonably distribute these points on each computing node, it is necessary to spatially cluster the Location data before allocating the data. The K-means algorithm is used, and the number of computing nodes is used as the number of clusters of the algorithm. The cluster number corresponding to the Location data is the target computing node to which it is to be allocated. The K-means algorithm can be completed locally where the data distributor is located. The algorithm is not complicated and can quickly obtain results in a short time without excessively affecting the duration of the partitioning process.

[0120] Optionally, the partitioning strategy includes a time strategy. Similar to Location data, if the spatiotemporal analysis to be performed is more affected by time, the K-means algorithm is used for event data Event to cluster the event data, and the cluster number corresponding to the Event data is the target computing node to which it is to be assigned.

[0121] In some embodiments, graph data formed by conversion and allocation of spatiotemporal data is stored in a memory storage.

[0122] In some embodiments, the distributed computing framework and communication framework are provided by the computing engine, the graph data in the memory storage is read, and the analysis algorithm in the algorithm library specified by the computing task is executed. Among them, the algorithm library has built-in multiple spatiotemporal analysis algorithms, some general spatiotemporal algorithms can be generated by simple extension of graph algorithms, and customized algorithms can be developed through the algorithm development interface of the graph computing system.

[0123] It can be seen that the transformation and optimization of the graph computing system is mainly achieved through the following aspects: data import, using the graph computing system to support the import of spatiotemporal data types; data partitioning, when doing spatiotemporal data analysis, the distributed data partitioning strategy of the graph computing system adds partitioning by spatial range and time, so that data close in space or time are divided into the same area to reduce network transmission; data structure, graph data modeling for spatiotemporal data, adding necessary vertex edge types and attributes, and storing time and space information; algorithm, enhancing the existing graph algorithm, adding input support for time and space parameters, and adding spatiotemporal operations to the calculation logic. In this way, the existing graph computing system can be transformed and optimized to support spatiotemporal data analysis, so as to achieve the goal of a computing system that is compatible with both spatiotemporal data and graph data analysis.

[0124] In this way, by transforming and optimizing the existing graph computing system, the existing graph computing system can be transformed and optimized from multiple aspects such as data import, data storage, data model, and algorithm, so that the existing graph computing system can support spatiotemporal data analysis. On the one hand, the data types supported by the graph computing system are expanded, and support for spatiotemporal data is increased. The distributed computing and communication framework of the graph computing system can be reused, and the existing graph algorithm can be slightly modified to perform high-performance spatiotemporal data analysis. On the other hand, from the user's perspective, the construction of multiple computing systems is avoided, saving software and hardware costs. The problem of enabling the existing graph computing system to support the analysis of spatiotemporal data is solved. For example, given a piece of spatiotemporal data, import it into the graph computing system and run the "spatial path analysis algorithm".

[0125] It should be noted that the above describes some embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the above embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0126] Based on the same technical concept, corresponding to any of the above-mentioned embodiments and methods, the present application also provides a graph computing system 500 supporting spatiotemporal data analysis. It should be noted that the graph computing system 500 supporting spatiotemporal data analysis is a distributed computing system including multiple candidate nodes.

[0127] like Figure 5 As shown, the graph computing system 500 supporting spatiotemporal data analysis may include: a data loader 501, a data partition and distributor 502, a memory storage 503, and a computing engine 504;

[0128] The data loader 501 is used to input the original spatiotemporal data into the graph computing system, and generate a plurality of target graph data corresponding to the plurality of original spatiotemporal data according to a preset graph data structure and a plurality of original spatiotemporal data in the candidate nodes, wherein the graph data structure is determined according to the analysis requirement information of the spatiotemporal data;

[0129] A data partitioner and distributor 502 is used to distribute a plurality of target graph data in the candidate nodes to target nodes matching the target graph data according to a target partitioning method, wherein the target partitioning method is determined according to the analysis requirement information, and store the plurality of target graph data in the memory storage 503 of the corresponding target node respectively;

[0130] The computing engine 504 is used to read the target graph data in the memory storage 503, call the target analysis algorithm in the pre-built algorithm library according to the target graph data and analysis requirement information, and execute the target analysis algorithm on the target graph data to obtain the analysis result.

[0131] Optionally, the raw spatiotemporal data is in the format of a comma-separated values ​​(CSV) file.

[0132] In some embodiments, the graph computing system 500 supporting spatiotemporal data analysis further includes a first determination module ( Figure 5 (not shown) is used to determine a target partitioning method that matches the analysis requirement information from among several preset candidate partitioning methods.

[0133] Optionally, the alternative partitioning methods include: hash partitioning, range partitioning, space partitioning and time partitioning;

[0134] The spatial partitioning includes: determining the number of candidate nodes in the graph computing system as the number of clusters, performing spatial clustering on the data, obtaining a target cluster number corresponding to the data, and the cluster number corresponds to the candidate node one by one; querying a matching target node from a number of candidate nodes according to the target cluster number;

[0135] The time partitioning includes: determining the number of candidate nodes in the graph computing system as the number of clusters, performing time clustering on the data, obtaining the target cluster number corresponding to the data, and the cluster number corresponds to the candidate node one by one; and querying the matching target node from several candidate nodes according to the target cluster number.

[0136] In some embodiments, the graph computing system 500 supporting spatiotemporal data analysis further includes a second determination module ( Figure 5 (not shown in the figure), which is used to split the alternative analysis algorithms in the algorithm library to obtain the general algorithm logic compatible with graph analysis and spatiotemporal analysis and the graph algorithm logic applied to graph analysis; determine the general algorithm logic as the target analysis base class algorithm, and determine the graph algorithm logic as the target analysis subclass graph algorithm; according to the target analysis base class algorithm, create a target analysis subclass spatiotemporal algorithm; determine the target analysis base class algorithm, the target analysis subclass graph algorithm and the target analysis subclass spatiotemporal algorithm as the target analysis algorithm, and store the target analysis algorithm in the algorithm library.

[0137] It should be noted that, for the convenience of description, the above device is described in terms of functions and is divided into various modules. Of course, when implementing the present application, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0138] The device of the above embodiment is used to implement the corresponding spatiotemporal data analysis method in any of the above embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be repeated here.

[0139] Based on the same technical concept, corresponding to any of the above-mentioned embodiment methods, the present application also provides an electronic device.

[0140] Figure 6 A more specific schematic diagram of the hardware structure of an electronic device provided by this embodiment is shown.

[0141] The electronic device 600 may include a processor 601 and a memory 602 storing computer program instructions.

[0142] Specifically, the processor 601 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0143] The memory 602 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 602 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 602 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 602 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 602 is a non-volatile solid-state memory.

[0144] In certain embodiments, the memory may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium device, an optical storage medium device, a flash memory device, an electrical, optical or other physical / tangible memory storage device. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of the present application.

[0145] The processor 601 implements any one of the spatiotemporal data analysis methods in the above embodiments by reading and executing computer program instructions stored in the memory 602 .

[0146] In some examples, the electronic device 600 may further include a communication interface 603 and a bus 610. Figure 6 As shown, the processor 601, the memory 602, and the communication interface 603 are connected via a bus 610 and communicate with each other.

[0147] The communication interface 603 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0148] Bus 610 includes hardware, software or both, and the parts of online data flow billing equipment are coupled to each other. For example, but not limitation, bus 610 may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-end bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 610 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0149] Exemplarily, the electronic device 600 may be a mobile phone, a tablet computer, a laptop computer, a PDA, an in-vehicle electronic device, an ultra-mobile personal computer (UMPC), a netbook, or a personal digital assistant (PDA).

[0150] Based on the same technical concept, corresponding to any of the above-mentioned embodiments, the present application also provides a non-transitory computer-readable storage medium. The computer-readable storage medium stores computer program instructions; when the computer program instructions are executed by the processor, any of the spatiotemporal data analysis methods in the above-mentioned embodiments is implemented. Examples of computer-readable storage media include non-transitory computer-readable storage media, such as portable disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0151] Based on the same technical concept, corresponding to any of the above-mentioned embodiments, the present application also provides a computer program product, which includes computer program instructions. In some embodiments, the computer program instructions can be executed by one or more processors of a computer so that the computer and / or the processor execute the spatiotemporal data analysis method. Corresponding to the execution subject corresponding to each step in each embodiment of the spatiotemporal data analysis method, the processor that executes the corresponding step may belong to the corresponding execution subject.

[0152] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0153] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0154] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0155] The above reference is according to the method of the embodiment of the present application, the flow chart of the device (system) and the computer program product and / or the block diagram described various aspects of the present application.It should be understood that each square box in the flow chart and / or the block diagram and the combination of each square box in the flow chart and / or the block diagram can be realized by computer program instructions.These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the realization of the function / action specified in one or more square boxes of the flow chart and / or the block diagram.Such a processor can be but is not limited to a general-purpose processor, a special-purpose processor, a special application processor or a field programmable logic circuit.It can also be understood that each square box in the block diagram and / or the flow chart and the combination of the square boxes in the block diagram and / or the flow chart can also be realized by the dedicated hardware that performs the specified function or action, or can be realized by the combination of dedicated hardware and computer instructions.

[0156] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A spatiotemporal data analysis method, characterized in that: The method is applied to a graph computing system, which is a distributed computing system; The method comprises: The following steps A and B are performed in parallel in each candidate node in the graph computing system: Step A: generating a plurality of target graph data corresponding to the plurality of original spatiotemporal data respectively according to a preset graph data structure and a plurality of original spatiotemporal data in the candidate nodes, wherein the graph data structure is determined according to the analysis requirement information of the spatiotemporal data; Step B: according to the target partitioning method, respectively allocating a number of target graph data in the candidate nodes to target nodes matching the target graph data, wherein the target partitioning method is determined according to the analysis requirement information; The following steps C are executed in parallel on each target node: Step C: calling a target analysis algorithm in a pre-built algorithm library according to the target graph data and analysis requirement information, and executing the target analysis algorithm on the target graph data to obtain an analysis result.

2. The method according to claim 1, characterized in that Before respectively allocating a plurality of target graph data in the candidate nodes to target nodes matching the target graph data according to the target partitioning method, the method further includes: Determine a target partitioning method that matches the analysis requirement information from among a plurality of preset candidate partitioning methods; The alternative partitioning methods include: hash partitioning, range partitioning, space partitioning and time partitioning; The spatial partitioning includes: determining the number of candidate nodes in the graph computing system as the number of clusters, performing spatial clustering on the data, obtaining a target cluster number corresponding to the data, and the cluster number corresponds to the candidate node one by one; querying a matching target node from a number of candidate nodes according to the target cluster number; The time partitioning includes: determining the number of candidate nodes in the graph computing system as the number of clusters, performing time clustering on the data, obtaining the target cluster number corresponding to the data, and the cluster number corresponds to the candidate node one by one; and querying the matching target node from several candidate nodes according to the target cluster number.

3. The method according to claim 1, characterized in that: Before calling the target analysis algorithm in the pre-built algorithm library according to the target graph data and the analysis requirement information, the method further includes: Split the alternative analysis algorithms in the algorithm library to obtain the general algorithm logic compatible with graph analysis and spatiotemporal analysis and the graph algorithm logic applied to graph analysis; Determine the general algorithm logic as the target analysis base class algorithm, and determine the graph algorithm logic as the target analysis subclass graph algorithm; According to the target analysis base class algorithm, create a target analysis subclass spatiotemporal algorithm; The target analysis base class algorithm, the target analysis subclass graph algorithm and the target analysis subclass spatiotemporal algorithm are determined as target analysis algorithms, and the target analysis algorithms are stored in an algorithm library.

4. The method according to claim 1, characterized in that The format of the original spatiotemporal data is a comma separated value (CSV) file.

5. A graph computing system supporting spatiotemporal data analysis, characterized in that: The graph computing system is a distributed computing system, including multiple candidate nodes; The graph computing system includes a data loader, a data partitioner and distributor, a memory storage, and a computing engine; The data loader is used to input the original spatiotemporal data into the graph computing system, and generate a plurality of target graph data corresponding to the plurality of original spatiotemporal data according to a preset graph data structure and a plurality of original spatiotemporal data in the candidate nodes, wherein the graph data structure is determined according to the analysis requirement information of the spatiotemporal data; The data partitioner and distributor is used to distribute the target graph data in the candidate nodes to the target nodes matching the target graph data according to the target partitioning method, wherein the target partitioning method is determined according to the analysis requirement information, and store the target graph data in the memory storage of the corresponding target nodes respectively; The computing engine is used to read the target graph data in the memory storage, call the target analysis algorithm in the pre-built algorithm library according to the target graph data and analysis requirement information, and execute the target analysis algorithm on the target graph data to obtain the analysis result.

6. An electronic device, characterized in that: The device comprises: a processor and a memory storing computer program instructions; when the processor calls the computer program instructions, the spatiotemporal data analysis method as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are called by a processor, the spatiotemporal data analysis method according to any one of claims 1 to 4 is implemented.

8. A computer program product, characterized in that When the instructions in the computer program product are executed by a processor of an electronic device, the electronic device executes the spatiotemporal data analysis method as described in any one of claims 1 to 4.

Images