Data processing method and apparatus, electronic device and computer-readable storage medium
By clustering multiple spatial surfaces of the focus areas, global indexes and local indexes are generated, thus solving the problem of timeliness and fine-grained query of massive spatiotemporal trajectory data in the existing technology, and achieving efficient and real-time AOI spatial surface trajectory data query.
Patent Information
- Application Number
- PCT/CN2024/099314
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-12-08
- Filing Date
- 2024-06-14
- Publication Date
- 2025-06-12
AI Technical Summary
When processing massive space-time trajectory data in the prior art, it is difficult to realize fine-grained and time-efficient AOI spatial surface trajectory data query, resulting in a coarse granularity and low time-efficiency query range, which cannot meet the business party's high time-efficiency and fine-grained query needs.
By clustering multiple spatial surfaces of the concern area, global indexes and local indexes are generated, and these indexes are used to obtain the corresponding trajectory data of specific AOI space from the full trajectory data, thereby realizing fine-grained and time-efficient data query.
It improves the query efficiency and real-timeness of AOI spatial surface trajectory data, provides business parties with higher timeliness and fine-grained data support, and promotes the rapid iteration and accuracy of map products.
Smart Images

Figure CN2024099314_12062025_PF_FP_ABST
Abstract
Description
Data processing method and device, electronic device, and computer-readable storage medium
[0001] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on December 8, 2023, with application number 202311686069.X and invention name “Data processing method and device, electronic device, computer-readable storage medium”, the entire contents of which are incorporated by reference in this disclosure. Technical Field
[0002] The present disclosure relates to the field of data processing technology, and in particular to mapping technology, indexing technology, trajectory positioning, Internet of Things technology, etc. Specifically, the present disclosure relates to a data processing method and device, an electronic device, and a computer-readable storage medium. Background Art
[0003] With the rapid development of Internet of Things technology, more and more mobile terminals are able to use GPS (Global Positioning System) to collect the location information of moving objects to realize positioning functions and depict the movement trajectory of objects.
[0004] These location data form a massive amount of spatiotemporal trajectory data. By effectively collecting and processing these spatiotemporal trajectory data, we can help quickly discover changes in roads in the real world, thereby achieving rapid iteration of map products and enhancing the accuracy and timeliness of maps in depicting the real world.
[0005] Summary of the Invention
[0006] The present disclosure provides a data processing method and apparatus, an electronic device, and a computer-readable storage medium.
[0007] According to a first aspect of the present disclosure, a data processing method is provided, the method comprising:
[0008] Obtain index data; the index data is obtained by clustering multiple spatial surfaces of the region of interest, the index data including a global index and multiple local indexes, the local index corresponding to the clustering result one-to-one, being a relationship index of one or more spatial surfaces of the region of interest included in the clustering result, and the global index being a relationship index of multiple clustering results;
[0009] According to the index data, trajectory data corresponding to at least one spatial surface of the region of interest is obtained from the full trajectory data.
[0010] According to a second aspect of the present disclosure, there is provided a data processing device, the device comprising:
[0011] An index reading module is configured to obtain index data; the index data is obtained by clustering multiple spatial surfaces of the region of interest, and the index data includes a global index and multiple local indexes. The local index corresponds one-to-one with the clustering result and is a relationship index of one or more spatial surfaces of the region of interest included in the clustering result. The global index is a relationship index of multiple clustering results.
[0012] The trajectory data processing module is configured to obtain, from the full amount of trajectory data, trajectory data corresponding to at least one spatial surface of the region of interest according to the index data.
[0013] According to a third aspect of the present disclosure, an electronic device is provided, including:
[0014] at least one processor; and
[0015] A memory communicatively connected to the at least one processor; wherein,
[0016] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor so that the at least one processor can perform the data processing method.
[0017] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the above-mentioned data processing method.
[0018] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the above data processing method when executed by a processor.
[0019] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.
[0021] FIG1 is a flow chart of a data processing method provided by an embodiment of the present disclosure;
[0022] FIG2 is a flowchart illustrating some steps of another data processing method provided by an embodiment of the present disclosure;
[0023] FIG3 is a flowchart illustrating some steps of another data processing method provided by an embodiment of the present disclosure;
[0024] FIG4 is a schematic diagram of a clustering result generated after clustering AOI spatial surfaces, and a global index and a local index generated based on the clustering result, in another data processing method provided by an embodiment of the present disclosure, when the global index and the local index are R-tree indexes;
[0025] FIG5 is a flowchart illustrating some steps of another data processing method provided by an embodiment of the present disclosure;
[0026] FIG6 is a schematic structural diagram of a data processing device provided by an embodiment of the present disclosure;
[0027] FIG7 is a block diagram of an electronic device for implementing the data processing method according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0028] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0029] AOI (Area of Interest)-based surface query and data is a commonly used trajectory query method. Especially in gate, internal or external road scenes, retrieving trajectories within the AOI surface for road excavation is an important means to solve AOI surface road excavation.
[0030] In some related technologies, full-volume filtering queries are performed based on national trajectories and AOI surfaces. However, this type of query processing has a coarse granularity and low timeliness, and cannot provide business parties with further high-timeliness queries and fine-grained queries (such as only querying the trajectory of a certain AOI surface).
[0031] The data processing method and device, electronic device, and computer-readable storage medium provided by the embodiments of the present disclosure are intended to solve at least one of the above technical problems in the prior art.
[0032] The data processing method provided in the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be an in-vehicle device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, an in-vehicle device, a wearable device, etc. The method can be implemented by a processor calling computer-readable program instructions stored in a memory. Alternatively, the method can be executed by a server.
[0033] Figure 1 shows a flow chart of a data processing method according to an embodiment of the present disclosure. As shown in Figure 1 , the data processing method according to an embodiment of the present disclosure may include steps S110 and S120.
[0034] In step S110, index data is obtained; the index data is obtained by clustering multiple spatial surfaces of the region of interest, and the index data includes a global index and multiple local indexes. The local index corresponds to the clustering result one by one, and is a relationship index of one or more spatial surfaces of the region of interest included in the clustering result. The global index is a relationship index of multiple clustering results.
[0035] In step S120 , trajectory data corresponding to at least one spatial surface of a region of interest is obtained from the full trajectory data according to the index data.
[0036] For example, the index data may be spatial index data, specifically index data corresponding to the AOI spatial surface (ie, the space corresponding to the AOI), which may be a relational index of the spatial relationship of the AOI spatial surface and may be obtained by clustering multiple AOI spatial surfaces.
[0037] The multiple AOI spatial surfaces may be multiple AOI spatial surfaces within a specific range, such as multiple AOI spatial surfaces across the country.
[0038] The basis for clustering may be the spatial distance between AOI spatial surfaces. Clustering may form one or more clustering results, and each clustering result includes one or more AOI spatial surfaces.
[0039] In some possible implementations, the spatial distance between the AOI spatial surfaces may be determined by the distance between the center points of the AOI spatial surfaces.
[0040] In some possible implementations, a local index may be generated based on the relationship between one or more AOI spatial surfaces included in a clustering result during the clustering process, and a global index may be generated based on the relationship between multiple clustering results during the clustering process.
[0041] In some possible implementations, clustering of multiple AOI spatial surfaces to generate index data may be performed in advance, and the generated index data may be stored in a specific storage space. In step S110, obtaining the index data is to read the stored index data from the specific storage space.
[0042] In some possible implementations, the full trajectory data can be a massive amount of spatiotemporal trajectory data generated based on the location information of mobile objects collected by systems such as GPS. The full trajectory data includes trajectory data corresponding to the AOI spatial surface, but does not establish a correspondence with the AOI spatial surface.
[0043] In some possible implementations, in step S120, obtaining trajectory data corresponding to the spatial surface of the area of interest from the full trajectory data according to the index data can be performed by filtering the full trajectory data according to the global index to obtain trajectory data corresponding to the local index, and obtaining trajectory data corresponding to multiple AOI spatial surfaces corresponding to the local index according to the trajectory data corresponding to the local index.
[0044] In some possible implementations, after the trajectory data corresponding to the AOI space surface is acquired, the trajectory data corresponding to the AOI space surface may be stored, so that the business party can determine the trajectory data corresponding to the AOI space surface by accessing the storage space.
[0045] In some possible implementations, the business party may have a computer program executor with map functions. That is, the trajectory data corresponding to the AOI spatial surface can be used for map updates. That is, after obtaining the trajectory data corresponding to the AOI spatial surface, the current changes in the roads of the AOI spatial surface in the real world can be discovered by processing the trajectory data corresponding to the AOI spatial surface, thereby achieving rapid iteration of map products and enhancing the accuracy and timeliness of the map's depiction of the world.
[0046] In some possible implementations, the data processing method provided by the embodiments of the present disclosure may be executed at predetermined intervals (such as daily or hourly) to achieve timely updates of trajectory data and provide business parties with more timely trajectory data.
[0047] In the data processing method provided in the embodiment of the present disclosure, the full amount of trajectory data is processed by index data, so that the trajectory data corresponding to a certain AOI spatial surface can be obtained from the full amount of trajectory data, thereby providing the business party with fine-grained data refined to the AOI spatial surface.
[0048] The data processing method provided by the embodiment of the present disclosure is described in detail below.
[0049] As described above, the index data can be obtained by clustering multiple AOI spatial surfaces. Before executing the step of obtaining the index data, the data processing method provided by the embodiment of the present disclosure may further include the step of clustering multiple AOI spatial surfaces to obtain the index data.
[0050] FIG2 is a flow chart showing a method for clustering multiple AOI spatial surfaces and obtaining index data.
[0051] As shown in FIG2 , clustering multiple AOI spatial surfaces and obtaining index data may include steps S210 , S220 , and S230 .
[0052] In step S210, a plurality of spatial surfaces of interest regions are clustered to obtain a plurality of clustering results; each clustering result includes one or more spatial surfaces of interest regions;
[0053] In step S220, a local index corresponding to each clustering result is generated according to the relationship between one or more spatial surfaces of the region of interest included in each clustering result during the clustering process;
[0054] In step S230 , a global index is generated according to the multiple clustering results.
[0055] In some possible implementations, in step S210 , clustering the multiple AOI space surfaces may be clustering the multiple AOI space surfaces using a clustering algorithm.
[0056] In some possible implementations, an agglomerative hierarchical clustering algorithm may be used to cluster multiple AOI spatial surfaces.
[0057] FIG3 shows a flowchart of an implementation method of clustering multiple AOI spatial surfaces using an agglomerative hierarchical clustering algorithm. As shown in FIG3 , clustering multiple AOI spatial surfaces using an agglomerative hierarchical clustering algorithm may include step S310 , step S320 , and step S330 .
[0058] In step S310, the plurality of spatial surfaces of the region of interest are divided into a plurality of clusters, and the clusters are marked as incomplete;
[0059] In step S320, the similarity distance between each two clusters in the plurality of clusters marked as incomplete is calculated, and the two clusters corresponding to the smallest similarity distance are merged into a new cluster;
[0060] In step S330 , when the number of the spatial surfaces of the region of interest included in the new cluster reaches a preset number, the cluster is marked as completed.
[0061] In some possible implementations, in step S310 , dividing the multiple AOI space surfaces into multiple clusters may be dividing the multiple AOI space surfaces into multiple clusters according to distances between the AOI space surfaces.
[0062] In some possible implementations, the distance between the AOI space surfaces may be determined based on the distance between the center points of the AOI space surfaces.
[0063] In some possible implementations, each AOI space surface can be treated as a cluster, and the cluster can be marked as incomplete. In some specific implementations, the center point of the AOI space surface can be used to represent the AOI space surface and participate in the cluster calculation.
[0064] In some possible implementations, in step S320 , the similarity distance between each two clusters may be calculated by using a standard deviation to calculate the similarity clustering between the two clusters, thereby improving cluster cohesion.
[0065] In some possible implementations, calculating the similarity distance between two clusters may be to use the standard deviation of the average value of the distance between the AOI spatial surface included in the first cluster and all AOI spatial surfaces in the second cluster as the similarity distance between the first cluster and the second cluster.
[0066] For example, the first cluster includes Na AOI spatial surfaces, and the second cluster includes Nb AOI spatial surfaces. The distances between these Na AOI spatial surfaces and Nb AOI spatial surfaces are calculated respectively, that is, a total of Na*Nb distances, and the standard deviation of the average values of these distances is used as the similarity distance between the first cluster and the second cluster.
[0067] In some specific implementations, the following formula may be used to calculate the similarity distance between the first cluster and the second cluster:
[0068] Where proximity(x,y)=√(x i -y i ) 2 +(x j -y j ) 2 , Na is the number of AOI spatial surfaces included in the first cluster, Nb is the number of AOI spatial surfaces included in the second cluster, x is the center point of the AOI spatial surface included in the first cluster, xi is its X-axis coordinate, and xj is its Y-axis coordinate; y is the center point of the AOI spatial surface included in the first cluster, yi is its X-axis coordinate, and yj is its Y-axis coordinate.
[0069] The smaller the similarity distance between two clusters, the higher the similarity between the two clusters. Therefore, merging the two clusters corresponding to the smallest similarity distance into a new cluster is essentially merging the two clusters with the highest similarity among all clusters into a new cluster.
[0070] In some possible implementations, in the process of clustering AOI spatial surfaces, in order to avoid excessive differences in the number of AOI spatial surfaces included in different clustering results, resulting in excessive differences in the sizes of multiple local indexes generated based on the clustering results, and further resulting in excessive differences in the amount of calculation required to obtain the trajectory data corresponding to different AOI spatial surfaces based on the index data, resulting in instability in the process of obtaining the trajectory data corresponding to the AOI spatial surfaces, the maximum number of AOI spatial surfaces included in each clustering result is set.
[0071] If the total number of AOI spatial surfaces is M, and it is expected that the M AOI spatial surfaces will be divided into N clustering results, the maximum number of AOI spatial surfaces contained in each clustering result can be set to P = [M / N].
[0072] In some possible implementations, in step S330, the preset number value is the maximum number of AOI surfaces that each clustering result may include. When the number of AOI surfaces included in a cluster reaches the maximum number of AOI surfaces that each clustering result may include, the cluster is marked as complete, and the cluster no longer participates in cluster merging, and the number of AOI surfaces it includes does not continue to increase.
[0073] At the same time, if the cluster no longer participates in the merging of clusters, the AOI space surface it includes will no longer change and can be regarded as a clustering result.
[0074] In some possible implementations, if the number of AOI spatial surfaces included in the new cluster reaches the set maximum number of AOI spatial surfaces included in each clustering result, the new cluster is not processed.
[0075] In some possible implementations, after the processing of the new cluster is completed, the marking status of all clusters is checked. If the marking status of a cluster is incomplete, steps S320 and S330 are repeated. If the marking status of all clusters is completed, it means that the clustering is completed, clustering is stopped, and one cluster is taken as a clustering result.
[0076] By using the above-mentioned agglomerative hierarchical clustering algorithm to cluster multiple AOI spatial surfaces, it is possible to avoid excessive differences in the number of AOI spatial surfaces included in different clustering results, which leads to excessive differences in the sizes of multiple local indexes generated based on the clustering results. In turn, the amount of calculation required to obtain the corresponding trajectory data of different AOI spatial surfaces based on the index data is too different, resulting in instability in the process of obtaining the trajectory data corresponding to the AOI spatial surfaces.
[0077] In some possible implementations, in step S220, generating a local index corresponding to the clustering result according to the relationship between one or more AOI spatial surfaces included in the clustering result during the clustering process may be generating the local index according to a merging process of the cluster clusters to which the AOI spatial surfaces belong during the clustering process.
[0078] For example, during the clustering process, AOI spatial surface A1 and AOI spatial surface A2 are merged into cluster A3, cluster A3 and cluster A4 are merged into cluster A5, and cluster A5 is a clustering result. The local index corresponding to the clustering result can be A5 including A3 and A4, and A3 including A2 and A1.
[0079] Since the basis for merging clusters during the clustering process is the similarity distance between clusters, the distances between the AOI spatial surfaces included in the merged clusters are relatively close. Therefore, the distances between AOI spatial surfaces at similar positions in the local index generated by the merging process of the clusters to which the AOI spatial surfaces belong during the clustering process are also close, which makes it convenient to determine the AOI spatial surfaces adjacent to each other according to the index.
[0080] In some possible implementations, in step S230, generating a global index based on the clustering result may be to use the spatial range covered by one or more AOI spatial surfaces included in the clustering result as the identifier of the clustering result, and generate a global index based on the identifiers of multiple clustering results.
[0081] In some possible implementations, generating a global index can be generating a global index based on the parallel relationship of multiple clustering results, determining the distance between the spatial ranges corresponding to different clustering results based on the identification of the clustering results, and placing clustering results with closer distances as closer index positions in the process of generating the global index.
[0082] A high-performance spatial indexing mechanism is built through global indexing and local indexing, which enables accurate query of AOI spatial surfaces and improves the query efficiency of AOI spatial surfaces.
[0083] After improving the query efficiency of the AOI spatial surface, the efficiency of obtaining the trajectory data corresponding to the AOI spatial surface will also be improved. Since the full amount of trajectory data presents real-time, sudden, volatile, and infinite characteristics, improving the efficiency of the trajectory data corresponding to the AOI spatial surface can improve the real-time nature of the obtained trajectory data and provide better services for the business side.
[0084] In some possible implementations, the local index and the global index are R-tree indexes.
[0085] FIG4 shows a schematic diagram of a clustering result generated after clustering AOI spatial surfaces when the global index and the local index are R-tree indexes, and a global index and a local index generated based on the clustering result.
[0086] As shown in Figure 4, during the clustering process, AOI spatial surfaces R6 and AOI spatial surfaces R7 are merged into cluster R4, AOI spatial surfaces R8 and AOI spatial surfaces R9 are merged into cluster R5, and clusters R4 and R5 are merged into cluster R1. Clusters R1, R2, and R3 are clusters marked as completed, that is, clustering results. The generated R-tree local index is shown in Figure 4, with R6, R7, R8, and R9 as leaf nodes, the previous node of R6 and R7 is R4, the previous node of R8 and R9 is R5, and the previous node of R4 and R5 is R1; the generated R-tree global index is shown in Figure 4, with R1, R2, and R3 as leaf nodes, and their previous node is R0.
[0087] In some possible implementations, after obtaining the index data, obtaining the trajectory data corresponding to the AOI spatial surface from the full trajectory data according to the index data can be performed by filtering the full trajectory data to obtain the trajectory data corresponding to the local index, and obtaining the trajectory data corresponding to multiple AOI spatial surfaces corresponding to the local index according to the trajectory data corresponding to the local index.
[0088] Figure 5 shows a flow chart of an implementation method for filtering the full amount of trajectory data, obtaining trajectory data corresponding to a local index, and obtaining trajectory data corresponding to multiple AOI spatial surfaces corresponding to the local index based on the trajectory data corresponding to the local index. As shown in Figure 5, filtering the full amount of trajectory data, obtaining trajectory data corresponding to a local index, and obtaining trajectory data corresponding to multiple AOI spatial surfaces corresponding to the local index based on the trajectory data corresponding to the local index may include step S510 and step S520.
[0089] In step S510, the full trajectory data is classified according to the global index to obtain regional trajectory data corresponding to at least one local index;
[0090] In step S520 , a matching operation is performed on at least one regional trajectory data and at least one focus region spatial surface according to at least one local index to obtain trajectory data corresponding to the at least one focus region spatial surface.
[0091] In some possible implementations, in step S510, the full amount of trajectory data is classified according to the global index. The full amount of trajectory data can be filtered and partitioned according to the identifier of the clustering result, that is, the spatial range covered by the AOI spatial surface included in the clustering result, to obtain regional trajectory data corresponding to one or more local indexes.
[0092] In some possible implementations, a Join operation can be used to obtain the regional trajectory data corresponding to a local index. For example, if the full trajectory data is R and the global index is I1, the R Join I1 instruction can be used to obtain the regional trajectory data corresponding to the local index.
[0093] In some possible implementations, in step S520, the regional trajectory data is matched with the AOI spatial surface according to the local index, and the trajectory data corresponding to the AOI spatial surface is obtained. The regional trajectory data can be filtered and partitioned according to the spatial range covered by the AOI spatial surface to obtain the regional trajectory data corresponding to the AOI spatial surface.
[0094] In some possible implementations, a Join operation is used to obtain the regional trajectory data corresponding to the AOI spatial surface. For example, if the regional trajectory data is R1 and the local index is I2, the R1 Join I2 instruction can be used to obtain the regional trajectory data corresponding to the AOI spatial surface.
[0095] In some possible implementations, the trajectory data may include multiple trajectory points.
[0096] Classifying the full trajectory data according to the global index and obtaining the regional trajectory data corresponding to the local index may be classifying the trajectory points included in the full trajectory data according to the global index and obtaining the trajectory points corresponding to the local index as the regional trajectory data.
[0097] Similarly, according to the local index, the regional trajectory data is matched with the spatial surface of the area of interest, and the trajectory data corresponding to the spatial surface of the area of interest can be obtained by binding multiple trajectory points included in the regional trajectory data with the AOI spatial surface according to the local index to obtain one or more trajectory points corresponding to the AOI spatial surface.
[0098] The operation of binding a track point to an AOI spatial surface may be performed based on the positional relationship between the track point and the area corresponding to the AOI spatial surface. That is, when the track point is located within the area corresponding to the AOI spatial surface, the track point is bound to the AOI spatial surface; when the track point is located outside the area corresponding to the AOI spatial surface, the track point is not bound to the AOI spatial surface.
[0099] In some possible implementations, after obtaining the trajectory data corresponding to the AOI space surface, the trajectory data corresponding to the AOI space surface may be stored so that the business party can determine the trajectory data corresponding to the AOI space surface by accessing the storage space.
[0100] In some possible implementations, when the local index and the global index are R-tree indexes, based on the "locality principle", the trajectory data corresponding to the leaf nodes of the R-tree index can be saved to the storage space corresponding to the upper-level nodes of the leaf nodes.
[0101] That is to say, the trajectory data corresponding to the AOI space surface can be stored in the storage space corresponding to the cluster to which the AOI space surface belongs. In this way, each time the trajectory data corresponding to the AOI space surface is loaded, the trajectory data corresponding to the AOI space surface belonging to the same cluster as the AOI space surface can be read in the same storage space.
[0102] In some possible implementations, considering that when the business party queries the trajectory data corresponding to a certain AOI spatial surface, there is a high probability that it will continue to query the trajectory data of the adjacent AOI surface, for any AOI spatial surface, in response to the query request for the trajectory data corresponding to the AOI spatial surface, the trajectory data corresponding to the upper-layer node of the leaf node corresponding to the AOI spatial surface is loaded into the running memory for query.
[0103] That is, when a query request for the trajectory data corresponding to the AOI space surface is received, the trajectory data corresponding to the AOI space surface belonging to the same cluster as the AOI space surface will be loaded into the running memory for query or acquisition by the business party.
[0104] Since the trajectory data corresponding to the leaf node is stored in the storage space corresponding to the node on the upper layer of the leaf node, the trajectory data corresponding to the AOI spatial surface belonging to the same cluster as the AOI spatial surface can be read in the same storage space. Therefore, the memory required to obtain the trajectory data corresponding to the AOI spatial surface belonging to the same cluster at one time is also less.
[0105] By loading the trajectory data corresponding to the AOI spatial surface belonging to the same cluster, the effect of loading multiple queries once can be achieved, reducing the time consumption of reading files.
[0106] In some possible implementations, the data processing method provided by the embodiments of the present disclosure can be executed at predetermined intervals (e.g., daily or hourly) to achieve timely updates of trajectory data and provide more timely trajectory data to business parties. Similarly, after obtaining the updated trajectory data, the stored trajectory data is also updated in a timely manner to achieve timely updates of the stored trajectory data.
[0107] Based on the same principle as the method shown in FIG1 , FIG6 shows a schematic structural diagram of a data processing device provided by an embodiment of the present disclosure. As shown in FIG6 , the data processing device 60 may include:
[0108] Index reading module 610 is used to obtain index data; the index data is obtained by clustering multiple spatial surfaces of the region of interest. The index data includes a global index and multiple local indexes. The local index corresponds to the clustering result one by one and is the relationship index of one or more spatial surfaces of the region of interest included in the clustering result. The global index is the relationship index of multiple clustering results.
[0109] The trajectory data processing module 620 is configured to obtain trajectory data corresponding to at least one spatial surface of a region of interest from the full trajectory data according to the index data.
[0110] In the data processing device provided in the embodiment of the present disclosure, the full amount of trajectory data is processed by index data, so that the trajectory data corresponding to a certain AOI spatial surface can be obtained from the full amount of trajectory data, and thus fine-grained data refined to the AOI spatial surface can be provided to the business party.
[0111] In some possible implementations, the trajectory data processing module 620 includes: a global trajectory unit, configured to classify the full trajectory data according to a global index and obtain regional trajectory data corresponding to at least one local index; and a local trajectory unit, configured to match the at least one regional trajectory data with at least one spatial surface of a region of interest according to the at least one local index and obtain trajectory data corresponding to the at least one spatial surface of the region of interest.
[0112] In some possible implementations, the full trajectory data includes multiple trajectory points; and the local trajectory unit is further configured to: bind the multiple trajectory points included in the at least one regional trajectory data to at least one focus region spatial surface based on at least one local index, and obtain one or more trajectory points corresponding to the at least one focus region spatial surface.
[0113] In some possible implementations, the data processing device further includes: a storage module, configured to save trajectory data corresponding to at least one region of interest spatial surface into a storage space corresponding to each region of interest spatial surface.
[0114] In some possible implementations, the trajectory data processing module 620 is further configured to obtain trajectory data corresponding to at least one spatial surface of the region of interest from the full trajectory data according to the index data at predetermined intervals.
[0115] In some possible implementations, the data processing device also includes: a clustering module, used to cluster multiple focus area spatial surfaces to obtain multiple clustering results; each clustering result includes one or more focus area spatial surfaces; a local index module, used to generate a local index corresponding to each clustering result according to the relationship between the one or more focus area spatial surfaces included in each clustering result during the clustering process; a global index module, used to generate a global index based on multiple clustering results.
[0116] In some possible implementations, the clustering module includes: an initialization unit, used to divide multiple focus area spatial surfaces into multiple cluster clusters, and mark the cluster clusters as incomplete; a distance calculation unit, used to calculate the similarity distance between each two cluster clusters in the multiple cluster clusters marked as incomplete, and merge the two cluster clusters corresponding to the smallest similarity distance into a new cluster cluster; a marking unit, used to mark the cluster cluster as completed when the number of focus area spatial surfaces included in the new cluster cluster reaches a preset number value; a stopping unit, used to return to the step of calculating the similarity distance between each two cluster clusters in the multiple cluster clusters marked as incomplete until all cluster clusters are marked as completed, and the cluster cluster is used as the clustering result.
[0117] In some possible implementations, the local index module is further configured to generate a local index corresponding to each clustering result according to a merging relationship between one or more focus region spatial surfaces included in each clustering result during the clustering process.
[0118] In some possible implementations, the initialization unit is further configured to: treat each spatial surface of the region of interest as a cluster, and mark the cluster as incomplete.
[0119] In some possible implementations, the distance calculation unit is further used to: use the standard deviation of the average value of the distance between all the spatial surfaces of the area of interest included in the first cluster and all the spatial surfaces of the area of interest included in the second cluster as the similarity distance between the first cluster and the second cluster; wherein the first cluster and the second cluster are clusters marked as incomplete; and the distance between different spatial surfaces of the area of interest is determined based on the distance between the center points of the different spatial surfaces of the area of interest.
[0120] In some possible implementations, the global index module includes: an identification unit, used to use the spatial range covered by one or more focus area spatial surfaces included in each clustering result as the identification of each clustering result; and a generation unit, used to generate a global index based on the identifications of multiple clustering results.
[0121] In some possible implementations, the local index and the global index are R-tree indexes.
[0122] In some possible implementations, the data processing device further includes: a loading module, configured to save the trajectory data corresponding to each leaf node of the R-tree index to a storage space corresponding to an upper-layer node of each leaf node.
[0123] In some possible implementations, the data processing device also includes: a request processing module for loading the trajectory data corresponding to the upper-layer node of the leaf node corresponding to any one of the multiple focus area spatial surfaces into the running memory for query in response to a query request for the trajectory data corresponding to the focus area spatial surface.
[0124] It is understood that the above-mentioned modules of the data processing device in the embodiment of the present disclosure have the function of implementing the corresponding steps of the data processing method in the embodiment shown in Figure 1. This function can be implemented by hardware, or it can be implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions. The above-mentioned modules can be software and / or hardware, and the above-mentioned modules can be implemented separately or integrated into multiple modules. For the functional description of each module of the above-mentioned data processing device, please refer to the corresponding description of the data processing method in the embodiment shown in Figure 1, and will not be repeated here.
[0125] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.
[0126] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.
[0127] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the data processing method provided in the embodiment of the present disclosure.
[0128] Compared with the existing technology, this electronic device processes the full amount of trajectory data through index data, thereby obtaining the trajectory data corresponding to a certain AOI spatial surface from the full amount of trajectory data, and can provide business parties with fine-grained data refined to the AOI spatial surface.
[0129] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the data processing method provided by the embodiment of the present disclosure.
[0130] Compared with the existing technology, this readable storage medium processes the full amount of trajectory data through index data, thereby obtaining the trajectory data corresponding to a certain AOI spatial surface from the full amount of trajectory data, and can provide business parties with fine-grained data refined to the AOI spatial surface.
[0131] The computer program product includes a computer program, and when the computer program is executed by a processor, it implements the data processing method provided in the embodiment of the present disclosure.
[0132] Compared with the existing technology, this computer program product processes the full amount of trajectory data through index data, thereby obtaining the trajectory data corresponding to a certain AOI spatial surface from the full amount of trajectory data, and can provide business parties with fine-grained data refined to the AOI spatial surface.
[0133] FIG7 shows a schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0134] As shown in Figure 7, device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of device 700 can also be stored in RAM 703. Computing unit 701, ROM 702, and RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to bus 704.
[0135] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.
[0136] The computing unit 701 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as the data processing method. For example, in some embodiments, the data processing method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the data processing method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the data processing method by any other appropriate means (e.g., by means of firmware).
[0137] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system comprising at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0138] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.
[0139] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0141] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.
[0142] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.
[0143] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.
[0144] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.
Claims
1. A data processing method, comprising: Obtaining index data; the index data is obtained by clustering multiple spatial surfaces of the region of interest, the index data includes a global index and multiple local indexes, the local index corresponds to the clustering result one by one, and is a relationship index of one or more spatial surfaces of the region of interest included in the clustering result, and the global index is a relationship index of multiple clustering results; According to the index data, at least one trajectory data corresponding to the spatial surface of the region of interest is obtained from the full amount of trajectory data.
2. The method according to claim 1, wherein: The acquiring, according to the index data, trajectory data corresponding to at least one spatial surface of the region of interest from the full amount of trajectory data comprises: Classifying the full amount of trajectory data according to the global index, and obtaining regional trajectory data corresponding to at least one of the local indexes; According to at least one of the local indexes, a matching operation is performed on at least one of the regional trajectory data and at least one of the focus region spatial surfaces to obtain trajectory data corresponding to at least one of the focus region spatial surfaces.
3. The method according to claim 2, wherein: The full trajectory data includes a plurality of trajectory points; the matching operation is performed on at least one of the regional trajectory data and at least one of the spatial surfaces of the region of interest according to at least one of the local indexes to obtain trajectory data corresponding to at least one of the spatial surfaces of the region of interest, including: According to at least one of the local indexes, a binding operation is performed on a plurality of trajectory points included in at least one of the regional trajectory data and at least one of the focus region spatial surfaces to obtain one or more trajectory points corresponding to at least one of the focus region spatial surfaces.
4. The method according to claim 1, wherein: After obtaining at least one trajectory data corresponding to the spatial surface of the region of interest from the full amount of trajectory data according to the index data, the method further includes: The trajectory data corresponding to at least one of the spatial surfaces of the region of interest are stored in the storage space corresponding to each of the spatial surfaces of the region of interest.
5. The method according to claim 4, wherein: The acquiring, according to the index data, trajectory data corresponding to at least one spatial surface of the region of interest from the full amount of trajectory data comprises: At predetermined intervals, at least one trajectory data corresponding to the spatial surface of the region of interest is obtained from the full amount of trajectory data according to the index data.
6. The method according to claim 1, wherein: Before obtaining the index data, the method further includes: Clustering the plurality of spatial surfaces of the region of interest to obtain a plurality of clustering results; each of the clustering results includes one or more spatial surfaces of the region of interest; Generating a local index corresponding to each clustering result according to a relationship between one or more spatial surfaces of the region of interest included in each clustering result in a clustering process; The global index is generated according to the multiple clustering results.
7. The method according to claim 6, wherein: The clustering of the plurality of spatial surfaces of the regions of interest to obtain the plurality of clustering results comprises: Dividing the plurality of spatial surfaces of the region of interest into a plurality of clusters, and marking the clusters as incomplete; Calculating the similarity distance between each two clusters in the plurality of clusters marked as incomplete, and merging the two clusters corresponding to the smallest similarity distance into a new cluster; When the number of the spatial surfaces of the region of interest included in the new cluster reaches a preset number value, marking the cluster as completed; Return to the step of calculating the similarity distance between every two clusters in the plurality of clusters marked as incomplete until all the clusters are marked as completed, and use the clusters as clustering results.
8. The method according to claim 7, wherein: The generating of the local index corresponding to each clustering result according to the relationship between the one or more spatial surfaces of the focus area included in each clustering result in the clustering process comprises: According to the one or more spatial surfaces of the region of interest included in each clustering result, The merging relationship in the process generates a local index corresponding to each clustering result.
9. The method according to claim 7, wherein: The dividing the plurality of spatial surfaces of the region of interest into a plurality of clusters and marking the clusters as incomplete comprises: Each spatial face of the region of interest is regarded as a cluster, and the cluster is marked as incomplete.
10. The method according to claim 7, wherein: The calculating the similarity distance between each two clusters in the plurality of clusters marked as incomplete comprises: Taking the standard deviation of the average value of the distances between all the spatial surfaces of the region of interest included in the first cluster and all the spatial surfaces of the region of interest included in the second cluster as the similarity distance between the first cluster and the second cluster; The first cluster and the second cluster are clusters marked as incomplete; and the distances between different spatial surfaces of the region of interest are determined according to the distances between the center points of different spatial surfaces of the region of interest.
11. The method according to claim 6, wherein: Generating the global index according to the plurality of clustering results comprises: Using the spatial range covered by one or more spatial surfaces of the region of interest included in each clustering result as an identifier of each clustering result; The global index is generated according to the identifiers of the multiple clustering results.
12. The method according to claim 6, wherein: The local index and the global index are R-tree indexes.
13. The method according to claim 12, wherein: After obtaining at least one trajectory data corresponding to the spatial surface of the region of interest from the full amount of trajectory data according to the index data, the method further includes: The trajectory data corresponding to each leaf node of the R-tree index is saved to each leaf node. The storage space corresponding to the nodes in the previous layer of the child nodes.
14. The method according to claim 12, wherein: After obtaining at least one trajectory data corresponding to the spatial surface of the region of interest from the full amount of trajectory data according to the index data, the method further includes: For any one of the plurality of spatial surfaces of the area of interest, in response to a query request for trajectory data corresponding to the spatial surface of the area of interest, the trajectory data corresponding to the upper layer node of the leaf node corresponding to the spatial surface of the area of interest is loaded into the running memory for query.
15. A data processing device, comprising: An index reading module is configured to obtain index data; the index data is obtained by clustering multiple spatial surfaces of the region of interest, the index data includes a global index and multiple local indexes, the local index corresponds to the clustering result one by one, and is a relationship index of one or more spatial surfaces of the region of interest included in the clustering result, and the global index is a relationship index of multiple clustering results; The trajectory data processing module is configured to obtain trajectory data corresponding to at least one spatial surface of the region of interest from the full amount of trajectory data according to the index data.
16. The device according to claim 15, wherein: The trajectory data processing module comprises: A global trajectory unit is configured to classify the full amount of trajectory data according to the global index, and obtain regional trajectory data corresponding to at least one of the local indexes; The local trajectory unit is configured to match at least one of the regional trajectory data with at least one of the focus region spatial surfaces according to at least one of the local indexes, and obtain trajectory data corresponding to at least one of the focus region spatial surfaces.
17. The device according to claim 16, wherein: The full trajectory data includes multiple trajectory points; The local trajectory unit is further configured to: bind multiple trajectory points included in at least one of the regional trajectory data with at least one of the focus region spatial surfaces according to at least one of the local indexes, and obtain one or more trajectory points corresponding to at least one of the focus region spatial surfaces.
18. The device according to claim 15, wherein: The data processing device further comprises: The storage module is configured to save the trajectory data corresponding to at least one of the spatial surfaces of the region of interest to the storage space corresponding to each of the spatial surfaces of the region of interest.
19. The device according to claim 18, wherein The trajectory data processing module is further configured to: At predetermined intervals, at least one trajectory data corresponding to the spatial surface of the region of interest is obtained from the full amount of trajectory data according to the index data.
20. The device according to claim 15, wherein: The data processing device further comprises: A clustering module is configured to cluster the plurality of spatial surfaces of the region of interest to obtain a plurality of clustering results; each of the clustering results includes one or more spatial surfaces of the region of interest; A local index module is configured to generate a local index corresponding to each clustering result according to the relationship between one or more spatial surfaces of the focus area included in each clustering result in the clustering process; The global index module is configured to generate the global index according to the multiple clustering results.
21. The device according to claim 20, wherein: The clustering module comprises: an initialization unit, configured to divide the plurality of spatial surfaces of the region of interest into a plurality of clusters, and mark the clusters as incomplete; A distance calculation unit, configured to calculate a similarity distance between each two clusters in the plurality of clusters marked as incomplete, and merge the two clusters corresponding to the smallest similarity distance into a new cluster; a marking unit, configured to mark the cluster as completed when the number of the spatial surfaces of the region of interest included in the new cluster reaches a preset number value; The stopping unit is configured to return to the step of calculating the similarity distance between each two clusters in the plurality of clusters marked as incomplete until all the clusters are marked as completed, and use the cluster as the clustering result.
22. The device according to claim 21, wherein The local index module is further configured to generate a local index corresponding to each clustering result according to a merging relationship between one or more spatial surfaces of the region of interest included in each clustering result during a clustering process.
23. The device according to claim 21, wherein The initialization unit is further configured to: regard each of the spatial surfaces of the region of interest as a cluster, and mark the cluster as incomplete.
24. The device according to claim 21, wherein The distance calculation unit is further configured to: use the standard deviation of the average value of the distances between all the spatial surfaces of the region of interest included in the first cluster and all the spatial surfaces of the region of interest included in the second cluster as the similarity distance between the first cluster and the second cluster; The first cluster and the second cluster are clusters marked as incomplete; and the distances between different spatial surfaces of the region of interest are determined according to the distances between the center points of different spatial surfaces of the region of interest.
25. The device according to claim 20, wherein: The global index module includes: an identification unit configured to use the spatial range covered by one or more spatial surfaces of the region of interest included in each clustering result as an identification of each clustering result; A generating unit is configured to generate the global index according to identifiers of the plurality of clustering results.
26. The device according to claim 20, wherein: The local index and the global index are R-tree indexes.
27. The device according to claim 26, wherein: The data processing device further includes: a storage module configured to save the trajectory data corresponding to each leaf node of the R-tree index to the storage space corresponding to the upper layer node of each leaf node.
28. The device according to claim 26, wherein The data processing device further comprises: A loading module is configured to load any one of the plurality of spatial planes of the region of interest. The focus area spatial surface, in response to a query request for querying the trajectory data corresponding to the focus area spatial surface, loads the trajectory data corresponding to the upper layer node of the leaf node corresponding to the focus area spatial surface into the running memory for query.
29. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 14.
30. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are configured to cause the computer to perform the method according to any one of claims 1-14.
31. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 14.
Citation Information
Patent Citations
Information pushing method and device
CN110213311A
Data processing method and server cluster
CN111159107A
Data storage, indexing and query method and system of meta-cosmic space server
CN114443914A
Track adjoint relationship mining method and system based on secondary spatio-temporal index
CN116361327A
Data processing method and device, electronic equipment and computer readable storage medium
CN117786237A
Cited By
Vector database index recommendation method and device, equipment and medium
CN120336331A