A spatiotemporal big data mining method based on multi-view algorithm

Abstract

The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the field of data mining, in particular to a kind of spatiotemporal big data mining method based on multi-view algorithm.The present application relates to the

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data mining, in particular to a spatiotemporal big data mining method based on a multi-view algorithm. BACKGROUND

[0002] With the rapid development of emerging information technologies such as intelligent sensing, Internet and Internet of Things, cloud computing, people's location, behavior, even physical characteristics, atmosphere, water quality, and every change of the environment have become data that can be sensed, recorded, stored, separated and utilized. With the development of satellite navigation positioning technology, space integration remote sensing technology, geographic information system technology, and computer and communication network technology, the collective characteristics and physical characteristics of the earth's surface have become geospatial data that can be sensed, recorded, stored, separated and utilized. As the object of data analysis and mining, spatiotemporal big data is analyzed and mined to obtain the trend of time change and the spatial distribution rule.

[0003] Due to the large amount of data, data diversity, users may experience unacceptable running time or spend a lot of effort on preprocessing. In addition to the number and variety, the high data speed relative to the absorption capacity is another computing challenge, and users often experience data loss. Moreover, without good data optimization and processing, search sharing chaos is easily caused in the process of data reading use. SUMMARY

[0004] The present application provides a spatiotemporal big data mining method based on a multi-view algorithm, aiming at solving the problems existing in the prior art.

[0005] The present application provides a spatiotemporal big data mining method based on a multi-view algorithm, comprising the following steps:

[0006] S1. Data collection: real-time acquisition of spatiotemporal big data corresponding to business traffic according to business characteristics as source data, establishment of a data management library, and storage of the source data in the data management library;

[0007] S2. Data cleaning and filtering: classifying the source data and setting data filtering rules to perform cleaning and filtering operations on the data;

[0008] S3. Information fusion: using a multi-view adaptive spatiotemporal graph network to convolve the source data and extract spatiotemporal features associated with business characteristics in the source data, and fuse the extracted spatiotemporal features with the business characteristics;

[0009] S4. Drawing pedigree: constructing a relationship graph according to the association between business characteristics and spatiotemporal characteristics, importing the source data into the relationship graph, and managing the data according to the relationship graph.

[0010] As a further improvement of the application, in the step S1, the source data is obtained from a data terminal, a network platform, and an application program.

[0011] As a further improvement of the application, the spatio-temporal big data includes but is not limited to: spatio-temporal reference data, GNSS and position trajectory data, geodetic and gravity measurement data, remote sensing image data, map data, spatial media data, inventory management data, and financial data.

[0012] As a further improvement of the application, in the step S2, the data cleaning operation includes:

[0013] The data with high similarity is deleted from the source data, the source data is compensated and defects are found in the data cleaning process, the incomplete data in the source data is supplemented by association, and the compensated source data is unified in specification.

[0014] As a further improvement of the application, in the step S2, the data filtering operation includes:

[0015] The data is filtered by a rule-based filter to remove abnormal data that does not meet the rules; the filtered data is subjected to spatio-temporal gridding processing to store the spatio-temporal grid code; and the data in the spatio-temporal grid code that does not meet the spatio-temporal constraint condition is filtered.

[0016] As a further improvement of the application, in the step S3, the process of fusing spatio-temporal features and business features based on a multi-view adaptive spatio-temporal graph network includes:

[0017] S31: The business features of the source data are mapped to multiple channels through a fully connected layer;

[0018] S32: The features mapped to the multiple channels are input into a bidirectional anisotropic graph learning module to obtain a time graph and a space graph;

[0019] S33: The time graph is convolved to extract global time correlation features of the time graph;

[0020] S34: The space graph is processed according to the global time correlation features to obtain multi-scale space features and long-short term time features of the space graph;

[0021] S35: The input business features, long-short term time features, and multi-scale space features are fused to obtain multi-scale global spatio-temporal features.

[0022] As a further improvement of the application, in the step S32, the formula for constructing the time graph and the space graph using the bidirectional anisotropic graph learning module is:

[0023] M 1 = tanh ( αE 1 θ 1 )

[0024] M 2 = tanh ( αE 2 θ 2 )

[0025] M corr = M 1 M 2 T

[0026] A = ReLU ( tanh ( βf L ( M corr )))

[0027] wherein, M 1 and M 2 represent an initial correlation matrix, tanh represents a hyperbolic tangent function, α and β both represent a set hyperparameter, E 1, E 2 represent a learnable graph encoding matrix, θ 1 and θ 2 both represent a weight matrix, M corr represents a correlation matrix, T represents a transpose, A represents a bi-asymmetric graph adjacency matrix, ReLU represents an activation function, f L represents L a layer bi-asymmetric function.

[0028] As a further improvement of the present application, the step S4 comprises:

[0029] According to the association relationship between the business characteristics and the space-time characteristics, target space-time big data layer metadata is acquired from the data management library; based on the target space-time big data layer metadata, a space-time big data layer node is generated, and corresponding space-time big data layer node relationships are determined according to the relationship graph generation instruction; according to the space-time big data layer node relationships and the space-time big data layer node, a space-time big data layer node relationship graph is constructed through a preset graph construction rule, and data management is performed according to the space-time big data layer relationship graph.

[0030] The beneficial effects of the present application are: by establishing a data management library to store a large amount of raw data, and by preprocessing the data through classification, cleaning and filtering, the data is preliminarily screened, the pressure of data processing in the subsequent data fusion process is reduced, and the workload of processing data quantity and types is reduced; the association between business characteristics and space-time characteristics is established by using a multi-view adaptive space-time graph network, the user's query and use can be facilitated through the association state between them, and the data can be conveniently consulted and retrieved in the later stage through the form of constructing a graph, thereby improving the efficiency of data management. BRIEF DESCRIPTION OF DRAWINGS

[0031] Figure 1 is a flowchart of the space-time big data mining method based on the multi-view algorithm of the present application. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical scheme and advantages of the present application more clear and explicit, the present application is further described in detail below in combination with the drawings and examples.

[0033] As shown in Figure 1 , the space-time big data mining method based on the multi-view algorithm of the present application comprises the following steps:

[0034] S1. Data collection: real-time acquisition of space-time big data corresponding to business traffic as source data according to business characteristics, establishment of a data management library, and import of the source data into the data management library for storage. The acquired data is imported into the established data management library, which facilitates real-time calling of the source data during the preprocessing process of the source data, avoids delay in data acquisition, and improves the efficiency of data processing.

[0035] In step S1, the source data acquisition method includes acquiring space-time big data of text, image and video from data terminals, network platforms and application programs. Due to the diversification of big data storage methods, more data quantity and data types can be more fully acquired through multiple channels of information approach and multiple formats of information files, so that more data information related to business characteristics can be more fully extracted.

[0036] The spatiotemporal big data includes, but is not limited to, spatiotemporal reference data, GNSS and position trajectory data, geodetic and gravity measurement data, remote sensing image data, map data, spatial media data, inventory management data and financial data.

[0037] Specifically, the spatiotemporal reference data can include time reference data and space reference data; the GNSS and position trajectory data can include GNSS reference station data and position trajectory data, which can include personal trajectory data, group trajectory data, traffic trajectory data, information flow trajectory data, logistics trajectory data and fund trajectory data; the geodetic and gravity measurement data includes geodetic control data, gravity field data and magnetic force data; the remote sensing image data can include satellite remote sensing image data, aerial remote sensing image data, ground remote sensing image data and underground sensing data, and the satellite remote sensing image data can include visible light image data, microwave remote sensing image data, infrared image data and laser radar scanning image data; the map data is various map and atlas data; and the spatial media data is digitalized text, graphics, images, sound, video, image and animation media data with spatial position characteristics changing over time.

[0038] S2. Data cleaning and filtering: classifying the source data and setting data filtering rules to perform cleaning and filtering operations on the data.

[0039] The data cleaning operation includes:

[0040] The data with more repetition and similarity in the source data is deleted, and in the data cleaning process, the source data is compensated for missing data, the incomplete data in the source data is supplemented by association, and the compensated source data is unified in specification.

[0041] The data preprocessing includes data cleaning, data compensation and data format processing and unification, so that the source data can be refined and standardized, and the obtained source data is processed, wherein the processing process is that the data with more repetition and similarity in the source data is deleted in the data cleaning, and in the data cleaning process, the source data is compensated for missing data, the incomplete data in the source data is supplemented by association, and then the compensated source data is unified in specification to ensure the consistency of the data flow format.

[0042] The data filtering operation includes:

[0043] The data is filtered by using a rule-based filter to eliminate abnormal data in the data that does not meet the rules; the abnormal data includes data with incomplete fields, data existing in a blacklist and data not meeting business rules, and the setting of the corresponding rule filter can adopt an existing common way. The data after the filtering is subjected to spatiotemporal gridding processing to be stored in a spatiotemporal grid code; and data in the spatiotemporal grid code that does not meet the spatiotemporal constraint condition is filtered.

[0044] By filtering common abnormal data and spatial-temporal gridding the data, the data is refined, a large amount of redundant basic collection log storage is reduced, the storage cost is reduced, and the data with representative value is retained, which lays a good foundation for subsequent spatial-temporal correlation mining work.

[0045] S3. Information fusion: adopt a multi-view adaptive spatio-temporal graph network to convolve and extract the spatio-temporal features associated with the business characteristics in the source data, and fuse the extracted spatio-temporal features with the business characteristics.

[0046] The process of fusing the spatio-temporal features and the business characteristics based on the multi-view adaptive spatio-temporal graph network includes:

[0047] S31: and the business characteristics of the source data are mapped to multiple channels through a fully connected layer.

[0048] S32: the features mapped to the multiple channels are input into a bidirectional anisotropic graph learning module to obtain a time graph and a space graph.

[0049] In step S32, the formula for constructing the time graph and the space graph using the bidirectional anisotropic graph learning module is:

[0050] M 1 = tanh ( αE 1 θ 1 )

[0051] M 2 = tanh ( αE 2 θ 2 )

[0052] M corr = M 1 M 2 T

[0053] A = ReLU ( tanh ( βf L ( M corr )))

[0054] wherein, M 1 and M 2 represent the initial correlation matrix, tanhdenotes a hyperbolic tangent function, α and β both denote a set hyperparameter, E 1, E 2 respectively denote a learnable graph encoding matrix, θ 1 and θ 2 both denote a weight matrix, M corr denotes a relevance matrix, T denotes a transpose, A denotes a bi-asymmetric graph adjacency matrix, ReLU denotes an activation function, f L denotes L a layer bi-asymmetric function.

[0055] S33: Convolve the time graph to extract global time correlation features of the time graph.

[0056] S34: Process the spatial graph according to the global time correlation features to obtain multi-scale spatial features and long-short term time features of the spatial graph.

[0057] S35: Fuse the input business features, long-short term time features and multi-scale spatial features to obtain multi-scale global spatio-temporal features.

[0058] Through the multi-view adaptive spatio-temporal graph network, deep information learning is performed on the one-way correlation, so that the learned directed graph structure has bidirectional asymmetry, thereby discovering the upstream and downstream relationships hidden in the road network or time graph. The time graph convolution module is used to extract the global correlation in the time dimension, and a group of convolution kernels of different sizes are used to extract the long-term and short-term local correlations in the time dimension. The graph structure obtained by using the bidirectional asymmetric graph learning model extracts the global and local correlations in the spatial dimension. Further, the relationship between the business features, time features and spatial features is established and fused.

[0059] S4. Draw a pedigree: according to the correlation relationship between the business features and the spatio-temporal features, construct a relationship graph, import the source data into the relationship graph, and perform data management according to the relationship graph.

[0060] Specifically, target spatio-temporal big data graph layer metadata is obtained from the data management library according to the correlation relationship between the business features and the spatio-temporal features; spatio-temporal big data graph layer nodes are generated based on the target spatio-temporal big data graph layer metadata, and corresponding spatio-temporal big data graph layer node relationships are determined according to a relationship graph generation instruction; a spatio-temporal big data graph layer node relationship graph is constructed according to the spatio-temporal big data graph layer node relationships and the spatio-temporal big data graph layer nodes through a preset graph construction rule, and data management is performed according to the spatio-temporal big data graph layer relationship graph.

[0061] According to the relationship between the space-time big data layer node and the space-time big data layer node, a data node relationship graph is constructed to realize the management of various big data. The space-time big data layer node relationship includes attribute relationship, spatial relationship and time relationship. The space-time big data node spatial relationship graph, the data node time relationship graph and the data node attribute relationship graph can be realized to manage the big data in the form of multiple graphs. After the graph is established, the user can be provided with a query service. According to the user's query instruction, the corresponding data node relationship graph is displayed. The technical problem that it is difficult to efficiently manage the relationship between various space-time big data, analyze and mine the space-time big data, and the user experience is not high is avoided. The management efficiency of various big data is improved, and the user experience is improved.

[0062] The above is a further detailed description of the present application in combination with specific preferred embodiments, and cannot be considered as limiting the specific implementation of the present application to these descriptions. For ordinary skilled persons in the technical field to which the present application belongs, some simple deductions or substitutions can be made without departing from the concept of the present application, and all of them should be considered as falling within the protection scope of the present application.

Claims

1. A spatio-temporal big data mining method based on multi-view algorithm, characterized in that, Comprise the following steps: S1. Data collection: according to the business characteristics of real-time acquisition corresponding to the business traffic space-time big data as source data, establish data management library, the source data is imported into data management library storage; S2. Data cleaning filter: the source data is classified, and the data filtering rule is set to execute cleaning and filtering operation on the data; S3. Information fusion: based on multi-view adaptive space-time graph network, the source data is convolved and the space-time characteristics associated with the business characteristics in the source data are extracted, and the extracted space-time characteristics are fused with the business characteristics; S4. Draw pedigree: according to the association between business characteristics and space-time characteristics, a relationship graph is constructed, the source data is imported into the relationship graph, and data management is carried out according to the relationship graph; The process of fusing space-time characteristics and business characteristics based on multi-view adaptive space-time graph network in step S3 comprises: S31: the business characteristics of the source data are mapped to multiple channels through a fully connected layer; S32: the characteristics mapped to the multiple channels are input into a bidirectional heterogeneous graph learning module to obtain a time graph and a space graph; S33: the time graph is convolved to extract the global time correlation characteristics of the time graph; S34: the space graph is processed according to the global time correlation characteristics to obtain multi-scale space characteristics and long-short term time characteristics of the space graph; S35: the input business characteristics, long-short term time characteristics and multi-scale space characteristics are fused to obtain multi-scale global space-time characteristics; The step S4 comprises: According to the association between business characteristics and space-time characteristics, target space-time big data layer metadata is obtained from the data management library; space-time big data layer nodes are generated based on the target space-time big data layer metadata, and the corresponding space-time big data layer node relationship is determined according to the relationship graph generation instruction; the space-time big data layer node relationship graph is constructed according to the space-time big data layer node relationship and the space-time big data layer node through the preset graph construction rule, and data management is carried out according to the space-time big data layer relationship graph. 2.The method of claim 1, wherein, In step S1, the source data is obtained from data terminals, network platforms and application programs. 3.The method of claim 2, wherein, The space-time big data includes but is not limited to: space-time reference data, GNSS and position trajectory data, geodetic survey and gravity and magnetic survey data, remote sensing image data, map data, space media data, inventory management data and financial data. 4.The method of claim 1, wherein, In step S2, the data cleaning operation comprises: The data with more repetition and similarity in the source data is deleted, the source data is compensated and the missing data is supplemented in the data cleaning process, and the compensated source data is unified in specification. 5.The method of claim 1, wherein, In step S2, the data filtering operation comprises: The data is filtered by using a rule-based filter to remove abnormal data that does not meet the rules; the filtered data is processed into space-time grid coding for storage; the data that does not meet the space-time constraint condition in the space-time grid coding is filtered. 6.The method of claim 1, wherein, In step S32, the formula for constructing the time graph and the space graph by using the bidirectional heterogeneous graph learning module is: M 1 tanh ( αE 1 θ 1 ) M 2 = tanh ( αE 2 θ 2 ) M corr = M 1 M 2 T A = ReLU ( tanh ( βf L ( M corr ))) wherein, M 1 and M 2 denote the initial correlation matrix, tanh denotes the hyperbolic tangent function, α and β both denote a set hyperparameter, E 1, E 2 denote the learnable graph encoding matrices, θ 1 and θ 2 both denote weight matrices, M corr denotes the correlation matrix, T denotes the transpose, A denotes the bi-anisotropic graph adjacency matrix, ReLU denotes the activation function, f L denotes L the layer bi-anisotropic function.

Images