Construction method and device of KD tree index based on real-time flow data dynamic balance, equipment and medium

Abstract

This invention relates to the field of data indexing technology, and discloses a method, apparatus, device, and medium for constructing a dynamically balanced KD-tree index based on real-time streaming data. The method includes determining a target trajectory stream; determining the initial spatial range of the root node of the initial KD-tree based on the geographical range of the target trajectory stream; dividing the initial spatial range of the root node into two initial subspaces using the coordinates of the first point of interest as the dividing point; performing a dynamic insertion operation for each target point of interest reached after the first point of interest; triggering a split operation for the leaf node when the number of points of interest stored in any leaf node exceeds a first preset capacity threshold; and monitoring the comprehensive imbalance index of the initial KD-tree in real time, triggering a global reconstruction process when the comprehensive imbalance index value exceeds a preset reconstruction threshold. This solution improves the balance of the KD-tree and query efficiency.

Description

Technical Field

[0001] This invention belongs to the field of data indexing technology, specifically relating to a method, apparatus, device, and medium for constructing a dynamic balanced KD-tree index based on real-time streaming data. Background Technology

[0002] With the rapid development of the mobile internet, a large amount of trajectory data is generated through portable smart devices (such as smartphones and smartwatches). This trajectory data contains a wealth of information about users, including their activities, travel modes, location choices, and preferences, which plays an important role in various fields. However, this trajectory data has the following characteristics: (1) information is implicit; (2) data is generated over time; and (3) data is generated at a high frequency. Trajectory data is obviously streaming data, so traditional data processing methods cannot be used directly.

[0003] Trajectory data is not the user's Points of Interest (POIs); therefore, POIs need to be extracted from the trajectory first. Current methods for POI extraction mainly include: spatial clustering-based POI extraction, time-based POI extraction, and spatial density-based POI extraction. However, due to the fluidity of streaming data and the inability to observe the entire data, coupled with the inherent limitations of these methods, we need to address this issue by comprehensively utilizing both time-based POI and spatial distribution characteristics to construct a new POI extraction method that can ensure real-time identification of user points of interest within a constantly changing data stream.

[0004] Kd-trees, as a classic data structure, play a crucial role in multidimensional data querying and management. Binary trees are relatively simple in structure, easy to implement, and have low storage and computational costs. However, in streaming data scenarios, due to the massive data volume and continuously increasing dimensions, traditional kd-tree methods face problems such as low query efficiency and tree imbalance. Figure 1 In the kd-tree instance on streaming data, D is a streaming dataset containing 6 samples, each sample has and Two dimensions. In timestamp This uses a kd-tree built on top of D, with a node capacity of 1, meaning the node will split when the number of samples in the current node exceeds 1. (Timestamp) At that time, four samples were added to D, resulting in The nodes continue to split, and the kd-tree is updated to... The maximum depth difference is 1, meaning the numbers are in equilibrium. And... The maximum depth difference is 3, compared and The arrival of new samples causes the tree to become unbalanced. In fact, in large streaming datasets, the data density distribution tends to stabilize. This leads to a situation where, over time, the number of samples in densely populated areas increases, causing the leaf nodes of the kd-tree to split infinitely. For example, in... Figure 1 Among the samples, locations 7, 8, and 10 have the highest data density. Over time, these locations will record an increasing amount of data. The leaf nodes recording the corresponding samples will continuously split. This phenomenon will eventually lead to an infinite increase in the imbalance of the kd-tree; the larger the dataset, the greater the imbalance. An extremely imbalanced kd-tree will result in excessively long query paths for some data, leading to low query efficiency.

[0005] Intuitively, the key to solving this problem lies in its imbalance, that is, real-time adjustment based on its imbalance. However, several issues arise: First, measuring imbalance traditionally uses the maximum depth difference, but setting a maximum depth difference threshold can lead to excessively frequent self-updates, necessitating a new measurement method. Second, streaming data is generated rapidly, requiring frequent monitoring of imbalance to keep it within acceptable limits; however, frequent monitoring is time-consuming, necessitating an efficient monitoring mechanism. Finally, after detecting imbalance, rebuilding the kd-tree renders the index unusable, meaning users must wait during updates, with the waiting time increasing over time. Therefore, a new mechanism is required. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the present invention aims to provide a method, apparatus, device, and medium for constructing a dynamically balanced KD-tree index based on real-time streaming data, thereby solving the problems of low KD-tree balance and low query efficiency in existing technologies.

[0007] According to one aspect of this application, a method for constructing a dynamically balanced KD-tree index based on real-time streaming data is disclosed, the method comprising: Determine the target trajectory flow, which includes multiple points of interest arranged based on time series points; The initial spatial range of the root node of the initial KD tree is determined based on the geographical range of the target trajectory flow. Using the coordinates of the first point of interest as the dividing point, the initial spatial range of the root node is divided into two initial subspaces, and an initial child node is created for each initial subspace. For each target interest point reached after the first interest point, a dynamic insertion operation is performed: in the current structure of the initial KD tree, the corresponding leaf node is determined for each target interest point, and the target interest point is stored in the corresponding leaf node; When the number of points of interest stored in any leaf node exceeds the first preset capacity threshold, a split operation is triggered for that leaf node, converting it into an internal node and creating two new leaf nodes based on its median splitting rule. The overall imbalance index of the initial KD tree is monitored in real time. When the overall imbalance index value exceeds the preset reconstruction threshold, a global reconstruction process is triggered. The global reconstruction process reconstructs a new balanced KD tree based on the interest points stored in all leaf nodes of the initial KD tree up to the trigger time.

[0008] According to another aspect of this application, a construction apparatus for a dynamically balanced KD-tree index based on real-time streaming data is also disclosed, the apparatus comprising: A target trajectory flow determination module is used to determine a target trajectory flow, wherein the target trajectory flow includes multiple points of interest arranged based on time series points; The initial spatial range determination module is used to determine the initial spatial range of the root node of the initial KD tree based on the geographical range of the target trajectory flow. The initial child node creation module is used to divide the initial spatial range of the root node into two initial subspaces using the coordinates of the first point of interest as the dividing point, and to create an initial child node corresponding to each initial subspace. The dynamic insertion operation execution module is used to perform a dynamic insertion operation for each target interest point reached after the first interest point: in the current structure of the initial KD tree, determine the corresponding leaf node for each target interest point and store the target interest point in the corresponding leaf node; The split operation triggering module is used to trigger a split operation for any leaf node when the number of points of interest stored in any leaf node exceeds a first preset capacity threshold, converting it into an internal node and creating two new leaf nodes based on its median splitting rule. The global reconstruction process triggering module is used to monitor the comprehensive imbalance index of the initial KD tree in real time. When the comprehensive imbalance index value exceeds the preset reconstruction threshold, the global reconstruction process is triggered. The global reconstruction process reconstructs a new balanced KD tree based on the interest points stored in all leaf nodes of the initial KD tree up to the trigger time.

[0009] According to another aspect of this application, an electronic device is also disclosed, the electronic device including a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to cause the electronic device to perform the steps of the construction method of dynamically balancing KD-tree index based on real-time streaming data as described in any of the preceding claims.

[0010] According to another aspect of this application, a computer-readable storage medium is also disclosed, on which instructions are stored, which, when executed by a processor, implement the steps of the method for constructing a dynamically balanced KD-tree index based on real-time streaming data as described in any of the preceding claims.

[0011] The present invention includes, but is not limited to, the following beneficial effects: (1) This solution effectively overcomes the defect of the traditional KD tree’s performance deteriorating rapidly under streaming insertion through the closed-loop mechanism of insertion-monitoring-reconstruction, ensuring a high efficiency level of range query. Moreover, this method abandons the traditional batch processing mode and can perform real-time processing and index updates on the streaming data points (points of interest) that arrive one by one, meeting the needs of application scenarios with high data timeliness requirements; (2) The use of a trigger-based monitoring strategy avoids a large number of unnecessary balance calculations. This intelligent monitoring mechanism reduces the computational overhead of the monitoring link without affecting the balance effect; (3) The invention innovatively proposes a comprehensive imbalance index, which comprehensively considers the depth balance and node fullness of the tree, overcomes the limitations of a single index (such as the maximum depth difference), and provides a more scientific and comprehensive quantitative standard for evaluating the health of the tree structure; (4) By introducing a caching mechanism of query copy and temporary cache tree, it ensures that the front-end query service is completely unaffected when the high-overhead task of global reconstruction is performed in the background, achieving zero interruption and greatly improving the availability of the system and user experience. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0013] Figure 1 This is a kd-tree instance graph on streaming data in existing technologies; Figure 2 A comparison chart of the degree of skewness in existing binary trees; Figure 3 This is a schematic diagram of key point mining based on spatiotemporal sliding window according to an embodiment of this application; Figure 4 This is a schematic diagram of the real-time query mechanism in an embodiment of this application; Figure 5 This is a comparison chart of the unique tag sets of POIs extracted by the two methods in the embodiments of this application; Figure 6 This is a comparison chart of the frequency distribution of POI tags extracted by two methods in the embodiments of this application; Figure 7 Spatial distribution maps of POIs in the target area extracted by two methods in embodiments of this application; Figure 8This is a comparison chart of the number of nodes of two methods in this application embodiment on four datasets; Figure 9 This is a comparison chart of the maximum depth difference ratio of two methods in this application embodiment on four datasets; Figure 10 The two methods of imbalance degree IBD in this application embodiments are for different dataset sizes and Comparison chart under different values; Figure 11 The two methods described in this application are as follows: A comparison chart of tree-building efficiency; Figure 12 The two methods described in this application are as follows: Below is a comparison chart of query efficiency; Figure 13 This is a flowchart illustrating the method for constructing a dynamically balanced KD-tree index based on real-time streaming data, as described in an embodiment of this application. Figure 14 This is a structural block diagram of the device for constructing a dynamically balanced KD-tree index based on real-time streaming data according to an embodiment of this application; Figure 15 This is a schematic diagram of the structure of an electronic device. Detailed Implementation

[0014] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] like Figure 13 The diagram shown is a flowchart of a method for constructing a dynamically balanced KD-tree index based on real-time streaming data, according to an embodiment of this application. (See attached diagram.) Figure 13 It includes the following steps: S1400. Determine the target trajectory flow, which includes multiple points of interest arranged based on time series points.

[0016] Specifically, this step involves responding to the received initial trajectory stream by performing a linear scan starting from its starting point. The initial trajectory stream contains a sequence of sampling points arranged in order of timestamps. For the current scanning point, a spatiotemporal sliding window is constructed, containing all subsequent points within a threshold spatial distance from the current scanning point. The time span of all points within the sliding window is then determined. If the time span reaches or exceeds a preset time threshold, a dwell event is identified. Further, the centroids of all sampling points within the sliding window are determined, and the coordinates of the centroid of each sampling point and its temporal context information are output as a point of interest. After identifying a point of interest, the scanning starting point is moved to the next point after the sliding window to continue scanning. If the time threshold is not reached, the scanning starting point is moved one point backward, and the above process is repeated until the trajectory stream ends. Finally, all output points of interest are sorted according to their time sequence to obtain the target trajectory stream.

[0017] S1402. Determine the initial spatial range of the root node of the KD tree based on the geographical range of the target trajectory flow.

[0018] S1404. Using the coordinates of the first point of interest as the dividing point, divide the initial spatial range of the root node into two initial subspaces, and create an initial child node corresponding to each initial subspace.

[0019] S1406. For each target interest point reached after the first interest point, perform a dynamic insertion operation: In the current structure of the KD tree, determine the corresponding leaf node for each target interest point and store the target interest point in the corresponding leaf node.

[0020] Specifically, this step involves obtaining the target interest point after the first interest point. Starting from the root node, the segmentation dimension is determined based on the depth information of the currently visited node. The coordinates of the target interest point in this segmentation dimension are compared with the segmentation value of the current node's spatial range. The target interest point is then recursively selected and routed to the corresponding child node until a leaf node is reached. The target interest point is then stored in the leaf node. The target interest point is any one of the interest points that subsequently reach the KD tree.

[0021] S1408. When the number of points of interest stored in any leaf node exceeds the first preset capacity threshold, a split operation is triggered for that leaf node, converting it into an internal node, and creating two new leaf nodes based on its median splitting rule.

[0022] Specifically, this step is as follows: First, identify leaf nodes whose number of stored streaming data points exceeds a preset capacity threshold as leaf nodes to be split. Then, determine the depth, spatial range, and internal data point set of the leaf nodes to be split. Determine the split dimension of the leaf nodes to be split based on the parity of their depth. Then, determine the median of the coordinate values ​​of the data point set on the split dimension. Using the median as the dividing point, divide the spatial range of the leaf nodes to be split into two new subspaces. Further, take any data point in the data point set as the target data point. Compare the target coordinate value of the target data point on the split dimension with the median. Assign the target data point to one of the two new subspaces. Finally, create a corresponding new leaf node for each new subspace, clear the data point set of the original leaf node, convert it into an internal node, and set its child node pointers to point to the two newly created leaf nodes.

[0023] S1410: Monitor the overall imbalance index of the KD tree in real time. When the overall imbalance index value exceeds the preset reconstruction threshold, trigger the global reconstruction process.

[0024] The global reconstruction process involves rebuilding a new balanced KD tree based on the interest points stored in all leaf nodes of the KD tree up to the trigger time.

[0025] Specifically, this step involves obtaining three key structural parameters of the KD tree: the current maximum depth, the current minimum depth, and the current total number of nodes. Based on these parameters, a comprehensive imbalance index is calculated. When the comprehensive imbalance index exceeds a preset static reconstruction threshold, a global reconstruction process is triggered.

[0026] Among them, the comprehensive imbalance index is the weighted sum of the node number ratio and the maximum depth difference ratio. The node number ratio is used to measure the fullness of the node space of the KD tree, and the maximum depth difference ratio is used to measure the balance of the depth distribution of the KD tree. The formula for the fullness of the node space is: ; The formula for depth distribution uniformity is: ; The formula for the comprehensive imbalance index is: ; In the formula, and The preset balance factor, and , Used to adjust the importance of node sparsity in the overall index. Used to adjust for the importance of depth variability in the overall index. The ratio of the number of nodes. For the maximum depth difference ratio, This represents the total number of actual nodes in the initial KD-tree. The number of nodes in a full binary tree. At the current maximum depth, This represents the minimum depth of the current tree. This is the current index.

[0027] Furthermore, upon meeting the triggering conditions, the global reconstruction process is automatically initiated. Subsequently, in response to the global reconstruction triggering conditions, the system traverses all leaf nodes of the current KD tree, collects all interest points stored in all leaf nodes of the current KD tree, constructs a static historical dataset, and, based on the static historical dataset, uses the median recursive segmentation algorithm to construct a new balanced KD tree. Specifically, this includes: initializing the new root node of the new balanced KD tree with the coordinate range of all interest points in the static historical dataset, and temporarily storing all interest points in the static historical dataset in the new root node. If the number of interest points stored in the new node... If the amount of data exceeds the preset node capacity threshold, the segmentation dimension is determined based on the depth of the new node. The new median of the coordinates of all interest points in the new root node on the segmentation dimension is determined. Using the new median as the segmentation point, the spatial range of the current new root node is divided, two new child nodes are created, and interest points are assigned to the corresponding new nodes according to their coordinates. The interest point assignment is recursively repeated for each newly created child node with excessive data volume until the number of interest points in all new nodes does not exceed the capacity threshold. The original KD tree is then switched to a newly constructed balanced KD tree, and the new balanced KD tree is set as the current valid index.

[0028] Specifically, the global reconstruction process is executed in conjunction with a caching mechanism. Before initiating the determination of the target trajectory flow, a copy of the KD tree is created as a query service copy. During the execution of the global reconstruction process, query requests are responded to based on the query service copy. During the execution of the global reconstruction process, all newly arrived points of interest are inserted into a temporary cache tree independent of the reconstruction process. After the switching step is completed, all points of interest in the temporary cache tree are merged and inserted into the newly constructed balanced KD tree. The query service copy and the temporary cache tree are destroyed, and the new balanced KD tree takes over the data insertion and query services.

[0029] Specifically, for ease of understanding, the specific embodiments of this application are described below: The formal definitions of the relevant concepts involved in this project are given below.

[0030] Definition 1 (GPS sampling point): A GPS sampling point is a user's location information obtained by a positioning device at a certain moment, denoted as . .in, It is the sampling point ID. It is the latitude and longitude information of the user's current location. It is the longitude information of the user's current location. This is the current timestamp.

[0031] Definition 2 (Trajectory): A trajectory is a set of GPS sampling points continuously generated by a user within a time period, denoted as ,in, It is the trajectory ID. It is the user ID. It is the set of GPS sample points contained in the trajectory, and the length of the trajectory is equal to... The number of sample points in the middle is denoted as .

[0032] Definition 3 (Point of Interest): A Point of Interest (POI) is a triple containing latitude, longitude, and a geographic label, denoted as... ,in, It is the POI ID. It's the latitude and longitude information of the POI. This is the longitude information of the POI. It is the geotag of POI.

[0033] Definition 4 (Trajectory Flow): A trajectory flow is a set of GPS sampling points generated by the user in real time. For a trajectory flow, only GPS sampling points with timestamps less than the current time are visible. That is, given the current time... ,for like ,but An invisible set of several trajectory flows is called a flow trajectory set, denoted as . .

[0034] Definition 5 (kd-tree): A kd-tree is a type of tree that... A binary tree index on dimensional data is defined as a triple. ,in It is a set of nodes. It is a node in a kd-tree. It is a set of edges. It is a virtual node representing the root of the kd-tree.

[0035] Typically, the maximum depth difference is used to measure the balance of a tree; that is, the difference in depth between the deepest and shallowest leaf nodes, denoted as . ,in , The depth of this node. The smaller the value, the more balanced the tree is considered to be.

[0036] However, this method of measurement can only reflect the balance of a tree to a certain extent. For example... Figure 2 As shown, Figure 2 (a) shows a skewed binary tree where the leftmost leaf node has a depth of 8, and the depths of all other leaf nodes are 3. It can be seen that... ; Figure 2 In the binary tree in (b), except for the rightmost leaf node which has a depth of 3, all other leaf nodes have a depth of 8. Therefore, its... However, the maximum depth difference is also 5, which shows... Figure 2 (a) binary and Figure 2 The degree of skewness in the binary trees in (b) is different. Therefore, using only... Using the same method to measure the balance of a tree is unreasonable. To address this issue, this project proposes to introduce a new method for measuring the balance of a kd-tree.

[0037] Definition 6 (Node Count Ratio): The node count ratio of a binary tree is defined as the ratio of its actual number of nodes to the total number of nodes when the tree is full (excluding leaf nodes), denoted as . ,in For the actual nodes of the binary tree, This represents the number of nodes when the binary tree is full. It is the maximum depth of the binary tree.

[0038] The node count ratio can reflect the skewness of nodes in a binary tree to some extent; that is, the closer a binary tree is to being full, the smaller its skewness. Next, we consider combining the node count ratio with the maximum depth difference to describe the imbalance of a binary tree. However, according to Definition 5, the node count ratio takes values ​​in the range [0,1], so we need to standardize the maximum depth difference first.

[0039] Definition 7 (Maximum Depth Difference Ratio): The maximum depth difference ratio of a binary tree is defined as its maximum depth difference divided by its maximum depth, denoted as . ,in It is the maximum depth difference. That is the maximum depth.

[0040] It can be seen that the maximum depth difference ratio also ranges from [0,1]. Therefore, both can be combined to measure the imbalance of a binary tree. A smaller node count ratio indicates a higher degree of tree imbalance; a larger maximum depth difference ratio indicates a higher degree of tree imbalance. Hence, the definition is as follows: Definition 8 (Imbalance of a Binary Tree): The imbalance of a binary tree is evaluated by combining its ratio of the number of nodes to its maximum depth difference, specifically defined as follows: ,in, , .

[0041] According to Definition 7, ,and The larger the value of , the higher the imbalance of the binary tree.

[0042] As mentioned earlier, when key points of a trajectory (POIs) are generated in real time as a stream, it can cause severe skewness in the kd-tree, thus affecting query efficiency. The purpose of this solution is to address the skewness problem in storing kd-trees containing POIs mined from trajectory streams.

[0043] Specifically, this solution first determines a target trajectory flow, such as... Figure 3 As shown, when user trajectories arrive in a streaming manner, sample points are sampled one by one. Considering that trajectory data arrives in a streaming manner and that GPS sampling intervals may be uneven, traditional density clustering methods face challenges in terms of real-time performance and semantic clarity. Therefore, this project adopts an efficient Stay-Point Mining Algorithm based on Spatio-Temporal Sliding Window to discover POIs in real time. This algorithm can effectively identify stay "events" in the trajectory that meet specific temporal and spatial constraints. The algorithm constructs a sliding window that meets a distance threshold D from each trajectory point through a linear scan, and checks whether its time span meets a time threshold T. If it does, the centroid of the point within the window is calculated as a POI, and the scanning starting point is moved outside the window; if it does not meet the requirements, the starting point is moved one position to the right and the scan continues until the trajectory ends.

[0044] As POIs are mined in real time, the project plans to store these POIs in a kd-tree in real-time as a stream. However, as mentioned earlier, this will inevitably cause skewness. According to Definition 7, this solution plans to monitor the BID value of the kd-tree in real time. If the BID value exceeds a given threshold... δ If the kd-tree is severely skewed, it needs to be restructured. During restructuring, consider using the current time point as a breakpoint. Do not add any more data to the kd-tree that needs restructuring. Treat all data before the current time point as historical data and rebuild the index using the traditional kd-tree method, making it a fully balanced tree. After restructuring, the new kd-tree will continue to receive data.

[0045] Furthermore, to ensure users can continue querying during kd-tree reconstruction, a caching mechanism is needed to guarantee that both old and new data are accessible. Understandably, new writes and queries cannot be performed during tree reconstruction. Assuming that at the current moment... If the value exceeds the threshold, the tree at the current time will be... Copy as, Then to Reconstruction will be carried out by Receive user queries; for newly arriving POI data, create a new cache tree. By storing them in this cache tree, users can simultaneously access... and Queries performed within the `<query>` element are completely unaffected by the refactoring. Once the reconstruction is complete, delete. ,Will Data storage In the middle, then delete This ensures that no data is lost and that users' real-time queries are not interrupted.

[0046] It is understandable that the kd-tree proposed in this scheme can dynamically maintain its balance, hence it is called a dynamically balanced kd-tree. Its structure is defined below.

[0047] Definition 9 (Dynamically Balanced kd-tree): A dynamically balanced kd-tree is based on the kd-tree in the streaming data S. and A binary tree with an index, each node , in and These are the node's ID and depth, respectively. It is a rectangle in two-dimensional space, that is The range of values, yes The vertex in the bottom left corner It is the top right corner vertex. If It is a leaf node. The coordinate range falls within The sample set within, and ,otherwise, Recorded Parent and child nodes, It is the ID of its parent node. It is the ID of the child node. If If it is a leaf node, then Each edge in E is represented as , root is ,in It does not contain a parent node.

[0048] When constructing a kd-tree, the first step is to determine the root node. This refers to the global range of sample values ​​in a kd-tree. Most kd-tree methods are designed for historical data, meaning the tree is built when the entire data is visible and its domain in each dimension is known. For streaming data, only data prior to the current time is available, making it impossible to predict future data; that is, the exact coordinate range of the entire tree cannot be determined from currently known data. However, in most cases, the range of values ​​can be inferred from the application scenario. For example, the longitude and latitude coordinate range of GPS streaming data for a city will not exceed the city's longitude and latitude range, and the temperature and humidity inside a greenhouse will not exceed the range monitored by the system. For these scenarios, their theoretical range can be directly used as the coordinate range. In the few scenarios where the theoretical range cannot be calculated, an approximate range can be inferred from historical data, and the coordinate range can be expanded when new data exceeds this range.

[0049] In a traditional kd-tree, determine Then, node splitting can begin from the root node. When splitting a node, a binary tree divides the space based on the median position of the samples in each dimension. Therefore, the number of samples in the two subspaces is roughly the same, resulting in roughly the same number of splits for each branch. This ensures a relatively balanced tree with varying leaf node depths, and thus this method is continued.

[0050] In historical data, root splitting is easy because all the data is visible, and the median can be easily found to split the data. However, in streaming data scenarios, from the initial timestamp... At this point, there is only one sample in the entire sample space, therefore there is no median. Therefore, when the first sample arrives, the first level of the binary tree is divided into two nodes based on its coordinates. Assume the coordinate range of the kd-tree is a rectangle. The coordinates of the first sample are ,here These are the coordinates of the bottom left corner of point R, i.e., the origin. These are the coordinates of the top right corner. From... The entire space is divided into two subspaces: Each subspace is stored in a node at the first level of the kd-tree. Subsequent samples are stored in these nodes based on their coordinates, until any node stores more than [a certain number of samples]. At this point, the nodes begin to split. It's worth noting that the coordinates of the first node are used to initially partition the binary tree because for large amounts of streaming data, the data density is essentially constant, meaning a node has a high probability of falling in the area of ​​highest density. Therefore, choosing the first node as the partitioning criterion makes it more likely to partition the space in areas of high data density and keep the initial kd-tree balanced for a longer period.

[0051] After the root split is complete, subsequent arriving samples are stored in the two child nodes according to their coordinates. When the number of samples stored in any node exceeds... At this point, the node begins to split. As shown above, each split does not divide the two-dimensional space into two equal subspaces. Instead, it is based on the median position of the sample points in the space. Specifically, it first finds all the sample points in the space. The median of the coordinates, and the point through which it passes. The axis divides the space into left and right parts. This results in two subspaces of unequal size and shape, which are the two child nodes resulting from the splitting of the node. In the next split, all sample points in this subspace are found. The median of the coordinates, and the point through which it passes. The x-axis divides the subspace into upper and lower parts. Each subsequent split begins again from the x-axis, and this process is repeated. The advantage of splitting from the median position is that each split begins at the point of highest sample density, resulting in a roughly equal number of samples in each subspace. This ensures maximum local balance in the kd-tree.

[0052] For ease of understanding, the splitting code is shown in Table 1: Table 1 Splitting Codes

[0053] Algorithm 1 is the splitting process. Steps 1 and 13 determine the splitting dimension based on the node depth, while steps 2-12 follow... Dimensional partitioning of child nodes, steps 14-24 according to Divide the dimensions into child nodes and assign the samples to the corresponding sample sets of the subspaces. Steps 25-32 store the results of the node splitting in the node and update the current node.

[0054] In the real-time kd-tree construction process, it is necessary to start from the first moment. Start monitoring the overall imbalance measurement of the tree The difficulty in the calculation lies in how to determine the number of nodes that reach the maximum depth in an ideal balanced tree.

[0055] The calculation has two parts: . The calculation requires the total number of nodes and the number of nodes if the binary tree is full; the latter can be calculated from the maximum depth of the tree. The calculation requires the maximum depth difference and the maximum depth of the tree. It can be seen that... and The calculation only requires the maximum depth, minimum depth, and number of nodes of the current tree. These values ​​can be recorded in real time using three variables in the program, and their values ​​can be directly called during calculation.

[0056] Algorithm 2 shows the process of calculating IBD, where steps 1-3 are calculated according to Definition 7. Steps 4-10 are calculated according to Definition 6. Step 11: Calculate the adjusted node ratio. Steps 12-15: Calculate according to Definition 8. .

[0057] Furthermore, for ease of understanding, Table 2 presents the calculation code for IBD as follows: Table 2 IBD Calculation Code

[0058] Once the IBD value exceeds the threshold δ, the kd-tree must be reconstructed. This is straightforward because the binary tree already contains a certain number of sample points. The reconstruction will use all points in the entire space as samples and split according to Algorithm 1. In practice, all data prior to the current timestamp can be seen during each reconstruction. The binary tree is split from the median position during the first split, therefore the binary tree obtained at the current timestamp must be balanced.

[0059] Algorithm 3 shows the process of reconstructing a kd-tree. Steps 1-8 are updated. The root node, where steps 5-7 directly segment the value range of the streaming dataset. Steps 9-23 split each node with a sample size greater than m to obtain the edge set. and node set .

[0060] Specifically, for ease of understanding, Table 3 presents the code for the reconstruction process:

[0061] Table 3 Codes for the Reconstruction Process

[0062] Based on the above process, a kd-tree completion algorithm is constructed, as shown in Algorithm 4. According to the coordinates of the first arriving sample in steps 1-5, the kd-tree is divided into two nodes. KD is initialized in steps 6-9. In steps 10-19, sequentially arriving points are inserted into KD. When the number of samples exceeds m, the node is split. Steps 20-23 evaluate the imbalance of KD; if... If the value exceeds δ, reconstruction will be performed.

[0063] Furthermore, for ease of understanding, Table 4 presents the code for creating a KD-tree:

[0064] Table 4 Code for Creating KD Numbers

[0065] A complete kd-tree creation algorithm is now available, but its efficiency in practice is relatively low. The time complexity of Algorithm 4 is analyzed, and the main reasons for its inefficiency are identified.

[0066] In Algorithm 4, samples from the streaming dataset S are inserted into KD one by one. Assume the streaming dataset is visible at the current time. The number of sample points is n, and the node capacity is m. Then the sample points... The insertion time complexity is This depends on the depth of the kd-tree. It can be seen that the sample insertion efficiency is very high.

[0067] When the number of samples in a node reaches m, the node splits, and the time complexity depends on the time required to find the median. In streaming data scenarios, sample points are sorted by time rather than by value; furthermore, when searching for the median, the samples in the node must first be sorted, resulting in a time complexity of O(m). The time complexity of finding the median in an ordered sequence is O(n log n). Therefore, the time complexity of obtaining the median at each node split is O(n log n). For ease of expression, the default base is 2 in the following text.

[0068] calculate The time complexity depends on the depth of KD, which is This is because in Algorithm 2, the number of iterations in the two loops equals the depth of KD (lines 6-9). It can be seen that a single calculation... It is also very efficient.

[0069] The time complexity of reconstructing KD depends on the number of node splits. Each split yields two child nodes, and the sample size of each child node is half that of its parent node (because it is split from the median position of the sample points). Therefore, the time complexity of the first split is O(log n). The time complexity of splitting all nodes in the second layer is O(n). More generally, the first The time complexity of splitting all nodes in the layer is O(n). Let the depth of the reconstructed kd-tree be j. Then the total time complexity is... ;in, Therefore, the time complexity of reconstructing KD is O(n log n). ;in Because the reconstructed binary tree is a balanced tree. Two reasons were found for the inefficiency of Algorithm 4: (1) the median needs to be found when splitting a node, and this operation needs to be repeated for each split; (2) although the calculation It is very efficient, but the process is repetitive. This reduces the efficiency of Algorithm 4; therefore, its efficiency decreases each time a new sample point arrives.

[0070] Furthermore, Algorithm 4 can be optimized to address the two issues mentioned above. The problem of finding the median during node splitting has two cases. First, before the first reconstruction of the kd-tree or between any two reconstructions, the complexity of each node split is O(n log n). (Including the quicksort algorithm), because splitting only occurs when the number of samples in a node exceeds m. Typically, m is small (e.g., 10-1000), so the efficiency of finding the median is not very low. Therefore, the algorithm for finding the median is not optimized in either of these two stages. Secondly, when the kd-tree triggers reconstruction, it is assumed that the visible dataset... It contains n sample points. The complexity of the first node split is O(n). Where n is relatively large. As time goes on, n increases, and the efficiency of finding the median decreases rapidly, and this process is repeated many times.

[0071] The core reason for the inefficiency of finding the median is sorting the data; therefore, sorting should be avoided to improve the efficiency of reconstruction. To this end, from the first moment, it is necessary to sort the data according to... Sort the sample points in each leaf node of the kd-tree. Therefore, maintain a list. Record the attributes of the samples in each leaf node respectively. The order of the nodes, where i is the node ID. , and use Replace this node In this way, each time a new sample point arrives, in addition to inserting it into a leaf node, the system is also updated. The time complexity is Because they are ordered, the total insertion time complexity of the sample points is O(n log n). Considering that m is a relatively small number, the insertion efficiency is still very high. When a reconstruction is triggered, all... Merge into a list and The time complexity is During each node split, The data is divided into two sublists based on the node's value R. Each sublist inherits its original order, and reordering is only required when a node splits into leaf nodes. This means the reconstruction algorithm only needs to sort the data once. Furthermore, because... Only store the ID of the stream data and its location. The value above, assuming the streaming data has k attributes, is stored. The space complexity is (Even if they are divided into multiple ), and in Algorithm 1 The same applies. Therefore, the volume of a kd-tree will not increase.

[0072] Furthermore, a set of improved splitting codes is provided in Table 5:

[0073] Table 5 Improved Splitting Code

[0074] The improved node splitting algorithm is shown in Algorithm 5. Steps 2 and 8 find the node by its position. and And its median. For an ordered sequence, the time complexity of finding the median by its position is O(n log n). Compared to steps 2-3 and 14-15 of Algorithm 1, the computational load is significantly reduced. In steps 5-6 and 11-12, by... and Divide the node into two sublists based on the median, so that they can be subsequently assigned to the corresponding two child nodes. It's important to note that these two sublists remain ordered. If the node... If there are m sample points, then calculate... and The time complexity of dividing into two sublists by the median is O(n). This is the same as steps 5-11 and 17-23 of Algorithm 1. Overall, Algorithm 5 is a significant improvement over Algorithm 1.

[0075] It's worth noting that there's an algorithm similar to quicksort that can be used to find the median, with an average time complexity of O(log n). However, in the context of this study, this method presents two problems. Firstly, its efficiency is affected by data quality; in the worst case, its complexity may reach [a certain level]. On the other hand, this method requires grouping the data and performing multiple iterations each time the median is found. In the splitting scenario of this study, this process needs to be repeated, leading to a significant decrease in overall efficiency. Therefore, a better approach is to sort the data once, thereby avoiding the performance loss caused by repeated calculations.

[0076] Regarding the second question above, through analysis The calculation process revealed that The value is only in the binary tree It only changes when the depth and depth difference change. The value will change when the number of nodes and the maximum depth of the binary tree change. This happens when the binary tree splits. When the depth remains unchanged, The value of will inevitably change. When the maximum depth of the tree changes, The value of will inevitably change because and Everything has changed.

[0077] When the maximum depth remains unchanged (i.e., the leaf node at the maximum depth does not split), the difference in maximum depth will remain regardless of how other nodes split. It will only decrease. Therefore, The value will only decrease, thus reducing The value of . In other words, when the maximum depth remains unchanged, the maximum depth difference Changes to the tree will not make it more unbalanced. This is true when the maximum depth remains constant but some nodes split. The value may decrease, but it rarely increases.

[0078] When new sample points continuously arrive and trigger node splits, but the maximum depth of KD remains unchanged, this usually indicates that the tree is likely moving towards a more balanced state. Therefore, it is possible to trigger a split only when a change in the maximum depth of KD is detected. Calculation, thus greatly reducing The number of calculations. Although there are rare cases where tree imbalance may go undetected, the probability of this happening is extremely low, and even if it does occur, the degree of imbalance exceeding the threshold will be very small.

[0079] Furthermore, experimental verification was conducted. The GPS trajectory dataset contained 17,621 trajectories generated by 182 users. To focus on POI mining in urban environments, the raw data was preprocessed and cleaned. Considering the concentration of the study area, geofencing was set to filter out trajectories located within the target area.

[0080] To evaluate the performance of our proposed method in POI mining, the DBSCAN algorithm, which is widely used in the field of POI mining, was selected as the baseline. DBSCAN is a classic density clustering algorithm that identifies high-density regions in space by connecting density-reachable points into clusters, thereby extracting POIs. It is widely used in the field of POI mining.

[0081] To ensure fairness in the comparative experiments, the key parameters of the two algorithms were strictly aligned. Spatially, the dwell window diameter D of our proposed method was set to 50 meters, while the neighborhood radius ε of DBSCAN was set to 25 meters to guarantee comparability on a spatial scale. Temporally, both methods used 1500 seconds (25 minutes) as the effective dwell time threshold. Considering the heterogeneity of the sampling intervals in the original datasets, a subset of trajectories with stable 5-second sampling intervals was selected for the experiments. Based on this, the min_samples parameter of DBSCAN was set to 300 original points, calculated from 1500 / 5 = 300, ensuring logical consistency with our proposed method in terms of both data volume and temporal significance.

[0082] Table 6 Comparison of experimental parameters and POI extraction results for the two methods

[0083] As shown in Table 6, the number of POIs extracted by our method (1179) is nearly ten times that of DBSCAN (129). This order-of-magnitude difference does not stem from the superiority or inferiority of one algorithm, but rather directly reflects the fundamental difference in their design philosophies: Our method is a fine-grained dwell point mining algorithm oriented towards individuals. As a streaming algorithm, it independently analyzes each trajectory, aiming to capture every effective dwell behavior of an individual user at different times and locations. For example, repeated dwellings of the same user in the school dormitory, library, and dormitory will be identified as three independent POIs. This mechanism makes the total number of POIs proportional to the complexity and frequency of the user's activities, resulting in highly personalized results. The DBSCAN algorithm, on the other hand, is a global, high-density public hotspot mining algorithm. As a batch clustering algorithm, DBSCAN aggregates all trajectory points of all users, aiming to identify public areas that are densely visited by numerous trajectories in space. Therefore, the repeated dwellings in the dormitory, library, and dormitory mentioned above will be identified as two POIs: dormitory and library. This results in a smaller number of extracted POIs, but each one represents a significant public hotspot.

[0084] To compare the mining performance of the two algorithms at the semantic level, this approach introduces the Jaccard Similarity Coefficient as one of the evaluation metrics. The Jaccard similarity coefficient is a classic metric for measuring the similarity between two finite sample sets, and its calculation formula is the size of the intersection of the two sets divided by the size of their union: ;in, These represent the sets of unique labels (geo-encoded) for all POIs extracted by the proposed method and the DBSCAN method, respectively.

[0085] The calculated Jaccard similarity coefficient between the two methods on the POI tag set is 0.8. This high value indicates that despite significant differences in underlying logic and processing modes, the two algorithms exhibit a high degree of consistency in identifying the semantic categories of core urban functional areas. Figure 5 We can also see that the two algorithms jointly discovered 12 identical POI tags, indicating that the streaming algorithm proposed in this scheme has comparable effectiveness to the classic DBSCAN method in identifying mainstream POI types. Furthermore, our method independently discovered 3 additional POI types that DBSCAN failed to recognize, and all 12 tag categories discovered by DBSCAN are completely covered by our method. This asymmetric inclusion relationship strongly demonstrates that our method has a greater ability to mine the breadth of POI semantic categories.

[0086] Figure 6 Comparison of tag frequency distribution and Figure 7 The spatial distribution map of the target area visually reveals the essential differences between the two algorithms. Within the target area, the proposed method, by independently analyzing each trajectory, accurately captures every effective stay of an individual in different buildings, thereby generating a large number of fine-grained personalized POIs (blue dots) while preserving complete temporal context. In contrast, DBSCAN aggregates the spatial information of all trajectory points while ignoring the temporal dimension, aiming to discover global hotspots. Therefore, it identifies the entire campus as a few POIs (red dots) such as dormitories, cafeterias, and classrooms, smoothing out individual behavioral differences. This also explains why DBSCAN can aggregate short-lived visits from multiple users at different times into a single POI, while the proposed method, due to not meeting the single-stay time threshold T, cannot identify such pseudo-stay points.

[0087] In summary, through multi-dimensional comparison with the classic DBSCAN algorithm, this section verifies the effectiveness of the proposed real-time POI mining method. Experimental results show that the proposed method can extract a large number of POIs reflecting users' personal behavioral habits from the original trajectory stream in real time and with fine granularity, demonstrating significant advantages in terms of quantity, semantic diversity, and personalized details.

[0088] To ensure fairness in the comparison, the baseline algorithm (traditional KD-tree) in this paper abandons the static bulk-loading method and instead adopts a point-by-point dynamic insertion method for construction, thereby simulating its natural evolution process in streaming data applications.

[0089] from and Within the coupling relationship, their different roles in the dynamic equilibrium mechanism can be further extracted. (Reconstructing the threshold) The system's tolerance for imbalance is defined, which determines when performance optimization should take place. The balance factor... This defines the system's sensitivity to imbalance, which determines the direction of optimization. In this example, the threshold will be reconstructed. The value was fixed at 0.5 to limit the tree's tolerance for imbalance, allowing it to grow naturally to a certain extent, thereby exploring the balance factor under uniform conditions. The impact on the growth morphology of dynamic KD-trees.

[0090] By the number of nodes and maximum depth difference ratio Two metrics were used to examine the performance of the two algorithms on four datasets of different sizes (with 951, 1903, 3806, and 7612 POIs, respectively).

[0091] Figure 8 The experimental results comparing the node count ratios of the two methods on four datasets clearly reveal the significant differences in tree structure between the two construction strategies. The traditional KD-Tree has a significantly lower node count ratio across all test datasets. All are constant at zero, indicating that in streaming insertion scenarios, this structure rapidly degenerates due to the lack of a balancing mechanism, resulting in a highly sparse and unbalanced topology. In contrast, dynamically balanced KD-Trees... Values ​​and weight parameters It exhibits a strong positive correlation. Given... A value closer to 1 represents better node space utilization and higher efficiency. Values ​​clearly help to build a fuller tree structure. However, The boundary conditions will cause the equilibrium criterion to completely ignore the depth difference. This could lead to an imbalance in the tree's height dimension. Therefore, A value between 0.5 and 0.75 is considered the best compromise, as it achieves high node utilization while still taking into full account depth balance.

[0092] Figure 9 The experimental results comparing the maximum depth difference ratios of the two methods on four datasets clearly reveal the significant differences between the two construction strategies in tree depth balance. The maximum depth difference ratio of the traditional KD-Tree on all test datasets... The height of these trees remains consistently high, around 0.8-0.9, indicating a highly uneven tree height and a risk of degenerating into a linear structure, severely impacting query efficiency. In contrast, dynamically balanced KD-Trees... Values ​​and weight parameters It exhibits a strong negative correlation. Given... A value closer to 0 indicates a better depth balance, while a higher value indicates a better depth balance. The value clearly helps to build a more depth-balanced tree structure. Although An increase in the value means that the center of gravity of the balance criterion shifts from Steering node ratio However, experimental results show that a tree with more complete nodes tends to achieve better depth balance in its macroscopic structure. In summary, and Keeping low It also takes into account the node ratio, so it performs relatively well in the depth dimension.

[0093] Experimental results show that It has a significant impact on the balance of a dynamically balanced KD-Tree. With... With the increase in [the value of the node], both the node number ratio and the maximum depth difference ratio show an overall improving trend. Taking into account both node distribution balance and depth balance, and It performs relatively well on four datasets and can be used as... Recommended values ​​for dynamically balancing KD-Trees.

[0094] To more comprehensively assess the overall balance of the tree, a comprehensive imbalance index was introduced. This indicator also takes into account and .

[0095] Figure 10 Imbalance between the two methods In different dataset sizes and The experimental results of the comparison plot under different values ​​comprehensively evaluate the performance of the two strategies in terms of overall tree balance. The traditional KD-Tree (blue solid line) is used as the baseline. The value consistently ranked near 1 in all experiments, indicating a significant disadvantage in node fullness. and depth balance It performed poorly in both dimensions, thus confirming the inefficiency of its overall structure. Furthermore, the traditional KD-Tree... The overall value shows an upward trend as the dataset size increases (from 951 to 7612), indicating that the increase in data volume does indeed exacerbate the imbalance problem caused by dynamic insertion of traditional KD-Tree, while the dynamic balance KD-Tree does not exhibit this problem.

[0096] Then, observe Extreme values: when (Only considering) )or (Only considering) )hour, The values ​​are typically at a high level. This confirms the initial assumption that a balancing strategy that only considers a single dimension (depth or saturation) has significant flaws.

[0097] Most notably, It exhibited stable and near-optimal performance across all four datasets. While it wasn't the absolute minimum at every data point, it consistently remained within the optimal range. The "valley" region of the curve. This strongly demonstrates that a curve based on node fullness... The dominant factor is 75% (with a weighting of 75%), while maintaining deep balance. As an auxiliary (weighted 25%) comprehensive strategy, it is the most robust and effective way to maintain the overall structural health of dynamic KD-trees and is the best choice for dealing with different data scales.

[0098] Based on the above analysis, it was determined that... The optimal balance factor Subsequently, an in-depth comparative analysis was conducted on the tree-building efficiency and query efficiency of dynamically balanced KD-trees and traditional static KD-trees.

[0099] Figure 11 The two methods in , The comparison chart of tree construction efficiency visually demonstrates the significant time cost difference between the two algorithms when building the index. The static KD-tree (solid blue line) has extremely low construction time; even on the largest dataset (7612 POIs), it only takes about 0.04 seconds. This fully demonstrates the efficiency advantage of its "batch processing" mode, which obtains all data at once through efficient global sorting and recursive partitioning. In contrast, the dynamically balanced KD-tree (dashed orange line), due to simulating point-by-point insertion and necessary dynamic reconstruction of the data flow, has a significantly higher time cost. On the largest dataset, its construction time reaches 83.9 seconds, thousands of times longer than the static tree. This overhead is the inevitable price paid for achieving real-time performance and adaptability to dynamic data flows, determined by the fundamental difference in the design philosophy of the two methods.

[0100] Query efficiency is the ultimate standard for measuring the performance of an index structure. Figure 12 The two methods in , The comparison of query efficiency reveals the core value of the dynamic balancing mechanism, with results that contrast sharply with tree construction efficiency: on a small dataset (951 POIs), the static tree performs faster. However, as the dataset size increases, the situation reverses. On the largest dataset (7612 POIs), the worst-case query time for the static tree rises to approximately 4.46 microseconds, while the dynamic balancing KD-tree takes only 3.21 microseconds, outperforming the static tree by about 28%. More importantly, the growth trends are also significant. The query time growth curve for the static tree (solid blue line) is steeper, indicating a certain risk of performance degradation as the data volume increases. The curve for the dynamic balancing KD-tree (dashed orange line) is flatter and more predictable. This strongly demonstrates that the dynamic reconstruction mechanism successfully suppresses structural degradation of the tree during continuous data insertion, ensuring that query performance remains highly stable even in the worst-case scenario.

[0101] Query efficiency is the ultimate standard for measuring the performance of an index structure. Figure 12 This study reveals the core value of the dynamic balancing mechanism, whose results stand in stark contrast to tree-building efficiency. To ensure fairness and challenge in the comparison, the query performance tests in this experiment were all conducted on the leaf nodes with the deepest depth in each tree structure. The average query time was obtained by performing thousands of queries on a single target point to obtain a stable and representative worst-case query time. The experimental results clearly show that the dynamic balancing mechanism sacrifices tree-building time for a significant advantage in query performance and stability. The query efficiency of the Standard KD-tree (solid blue line) deteriorates sharply with the increase in dataset size. Its time spikes from approximately 3.6 microseconds initially to approximately 6.4 microseconds on the largest dataset (7612 POIs), showing a steep growth curve. This typically reflects the continuous degradation of the tree structure during uninterrupted streaming insertion, leading to increasingly longer deepest paths and an exponential decline in query performance. In stark contrast, the query time of the dynamically balanced KD-tree (dashed orange line) exhibits excellent stability. Its execution time remained consistently at an extremely low level of 2.0 to 2.5 microseconds, with a nearly horizontal growth curve. This demonstrates that the dynamic reconstruction mechanism successfully suppressed the structural degradation of the tree during continuous data insertion, ensuring that query performance under worst-case conditions remained at an efficient and predictable constant level through periodic global optimization. It is less affected by the increase in data scale.

[0102] It is worth emphasizing that although dynamic tree construction takes longer numerically than static construction, this should be considered within the context of the entire streaming process. POI mining is inherently a continuous and time-consuming process, and the cost of building a dynamic tree is amortized over each real-time POI generation. Its millisecond-level single insertion overhead is perfectly acceptable for real-time systems. Therefore, this strategy of trading reasonable incremental costs for superior query performance is highly feasible in practical applications.

[0103] Furthermore, Figure 14 This is a structural block diagram of the apparatus for constructing a dynamically balanced KD-tree index based on real-time data streams, as described in the embodiments of the application. Figure 14 As shown, the device includes: The target trajectory flow determination module is used to determine the target trajectory flow, which includes multiple points of interest arranged based on time series points. The initial spatial extent determination module is used to determine the initial spatial extent of the root node of the KD tree based on the geographical extent of the target trajectory flow. The initial child node creation module is used to divide the initial spatial range of the root node into two initial subspaces using the coordinates of the first point of interest as the dividing point, and to create the initial child node corresponding to each initial subspace. The dynamic insertion operation execution module is used to perform dynamic insertion operations for each target interest point that arrives after the first interest point: in the current structure of the KD tree, determine the corresponding leaf node for each target interest point and store the target interest point in the corresponding leaf node; The split operation triggering module is used to trigger a split operation for any leaf node when the number of points of interest stored in any leaf node exceeds a first preset capacity threshold, converting it into an internal node and creating two new leaf nodes based on its median splitting rule. The global reconstruction process trigger module is used to monitor the comprehensive imbalance index of the KD tree in real time. When the comprehensive imbalance index value exceeds the preset reconstruction threshold, the global reconstruction process is triggered. The global reconstruction process reconstructs a new balanced KD tree based on the interest points stored in all leaf nodes of the KD tree up to the trigger time.

[0104] The application of the relevant modules of the device in this example can be referred to the relevant introduction of the method principle above, and will not be repeated here.

[0105] above Figure 14 The construction device for dynamically balancing KD-tree indexes based on real-time streaming data in this embodiment of the invention will be described in detail from the perspective of modular functional entities. The electronic device in this embodiment of the invention will be described in detail from the perspective of hardware processing.

[0106] Figure 15This is a schematic diagram of the structure of an electronic device 1600 provided in an embodiment of the present invention. The electronic device 1600 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 1611 (e.g., one or more processors) and a memory 1620, and one or more storage media 1630 (e.g., one or more mass storage devices) for storing application programs 1633 or data 1632. The memory 1620 and storage media 1630 can be temporary or persistent storage. The program stored in the storage media 1630 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the electronic device 1600. Furthermore, the processor 1611 may be configured to communicate with the storage media 1630 and execute the series of instruction operations in the storage media 1630 on the electronic device 1600.

[0107] Electronic device 1600 may also include one or more power supplies 1640, one or more wired or wireless network interfaces 1650, one or more input / output interfaces 1660, and / or one or more operating systems 1631, such as Windows Server, MacOSX, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 15 The illustrated electronic device structure does not constitute a limitation on electronic devices and may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0108] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of any of the above-described methods for constructing a dynamically balanced KD-tree index based on real-time streaming data.

[0109] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0110] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0111] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for constructing a dynamically balanced KD-tree index based on real-time streaming data, characterized in that, The method includes: Determine the target trajectory flow, which includes multiple points of interest arranged based on time series points; The initial spatial range of the root node of the initial KD tree is determined based on the geographical range of the target trajectory flow. Using the coordinates of the first point of interest as the dividing point, the initial spatial range of the root node is divided into two initial subspaces, and an initial child node is created for each initial subspace. For each target interest point reached after the first interest point, a dynamic insertion operation is performed: in the current structure of the initial KD tree, the corresponding leaf node is determined for each target interest point, and the target interest point is stored in the corresponding leaf node; When the number of points of interest stored in any leaf node exceeds the first preset capacity threshold, a split operation is triggered for that leaf node, converting it into an internal node and creating two new leaf nodes based on its median splitting rule. The overall imbalance index of the initial KD tree is monitored in real time. When the overall imbalance index value exceeds the preset reconstruction threshold, a global reconstruction process is triggered. The global reconstruction process reconstructs a new balanced KD tree based on the interest points stored in all leaf nodes of the initial KD tree up to the trigger time.

2. The method for constructing a dynamically balanced KD-tree index based on real-time streaming data according to claim 1, characterized in that, For each target interest point reached after the first interest point, a dynamic insertion operation is performed: in the current structure of the initial KD tree, determining the corresponding leaf node for each target interest point and storing the target interest point in the corresponding leaf node includes: After obtaining the first point of interest, the target point of interest is determined starting from the root node based on the depth information of the currently visited node. The coordinates of the target point of interest in this segmentation dimension are compared with the segmentation value of the current node's spatial range. The target point of interest is then recursively selected and routed to the corresponding child node until a leaf node is reached. The target point of interest is then stored in the leaf node. The target point of interest is any one of the subsequent points of interest that reach the initial KD tree.

3. The method for constructing a dynamically balanced KD-tree index based on real-time streaming data according to claim 1, characterized in that, The determination of the target trajectory flow includes: In response to the received initial trajectory stream, a linear scan is performed starting from the starting point of the initial trajectory stream, which contains a sequence of sampling points arranged in order of timestamps; For the current scan point, construct a spatiotemporal sliding window that includes all subsequent points starting from the current scan point and whose spatial distance is within the threshold range; Determine the time span of all points within the sliding window. If the time span reaches or exceeds a preset time threshold, then determine that a dwell event has occurred. Determine the centroid of all sampling points within the sliding window, and output the coordinates of the centroid of each sampling point and its temporal context as a point of interest; After identifying a point of interest, the scanning start point is moved to the next point after the sliding window to continue scanning; If the time threshold is not reached, the scanning starting point will be moved back one point, and the above process will be repeated until the trajectory flow ends. The target trajectory flow is obtained by sorting all the points of interest in the output according to the time point sequence.

4. The method for constructing a dynamically balanced KD-tree index based on real-time streaming data according to claim 1, characterized in that, When the number of streaming data points stored in any leaf node exceeds a preset capacity threshold, a split operation is triggered for that leaf node, converting it into an internal node and creating two new leaf nodes based on its median splitting rule, including: Determine the depth, spatial range, and internal data point set of the leaf node to be split, wherein the leaf node to be split is a leaf node whose number of stored stream data points exceeds a preset capacity threshold; The split dimension of the leaf node to be split is determined based on the parity of the depth of the leaf node to be split. Determine the median of the coordinate values ​​of the data point set along the split dimension; Using the median as the dividing point, the spatial range of the leaf node to be split is divided into two new subspaces; Based on the comparison between the target coordinate value and the median of the target data point in the split dimension, the target data point is assigned to one of two new subspaces, wherein the target data point is any data point in the set of data points; Create a new leaf node for each new subspace, clear the data point set of the original leaf node, convert it into an internal node, and set its child node pointers to point to the two newly created leaf nodes.

5. The method for constructing a dynamically balanced KD-tree index based on real-time data stream according to claim 1, characterized in that, The real-time monitoring of the overall imbalance index of the initial KD tree, and the triggering of the global reconstruction process when the overall imbalance index value exceeds a preset reconstruction threshold, includes: Obtain the current key structural parameters of the initial KD tree, wherein the current key structural parameters include the current maximum depth, the current minimum depth, and the current total number of nodes; Based on the aforementioned key structural parameters, calculate the comprehensive imbalance index. The comprehensive imbalance index is a weighted sum of the node number ratio and the maximum depth difference ratio. The node number ratio is used to measure the fullness of the node space of the initial KD tree, and the maximum depth difference ratio is used to measure the balance of the depth distribution of the initial KD tree. The formula for the fullness of the node space is: ; The formula for depth distribution uniformity is: ; The formula for the comprehensive imbalance index is: ; In the formula, and The preset balance factor, and , Used to adjust the importance of node sparsity in the overall index. Used to adjust for the importance of depth variability in the overall index. The ratio of the number of nodes. For the maximum depth difference ratio, This represents the total number of actual nodes in the initial KD-tree. The number of nodes in a full binary tree. At the current maximum depth, This represents the minimum depth of the current tree. For the current index; When the overall imbalance index exceeds the preset static reconstruction threshold, the global reconstruction process is triggered.

6. The method for constructing a dynamically balanced KD-tree index based on real-time data streams according to claim 1 or 5, characterized in that, The method further includes: In response to the triggering conditions of the global reconstruction process, all leaf nodes of the current KD tree are traversed, and all interest points stored in all leaf nodes of the current KD tree are collected to construct a static historical dataset; Based on the aforementioned static historical dataset, a new balanced KD tree is constructed using the median recursive partitioning algorithm, including: The new root node of the new balanced KD tree is initialized with the coordinate range of all points of interest in the static historical dataset, and all points of interest in the static historical dataset are temporarily stored in the new root node; If the number of interest points stored in the new node exceeds the preset node capacity threshold, then the segmentation dimension is determined according to the depth of the new node, and the new median of the coordinate values ​​of all interest points in the new root node on the segmentation dimension is determined. Using the new median as the dividing point, the spatial range of the current new root node is divided, two new child nodes are created, and the points of interest are assigned to the corresponding new nodes according to their coordinates. Recursively assign interest points to each newly created child node whose data volume exceeds the limit until the number of interest points in all new nodes does not exceed the capacity threshold. Switch the initial KD tree to the newly constructed balanced KD tree, and set the new balanced KD tree as the current valid index.

7. The method for constructing a dynamically balanced KD-tree index based on real-time data streams according to claim 6, characterized in that, The global reconstruction process is executed in conjunction with a caching mechanism, including: Before initiating the determination of the target trajectory flow, a copy of the initial KD tree is created as a copy of the query service; During the global reconstruction process, query requests are responded to based on a query service replica; During the global reconstruction process, all newly arrived points of interest are inserted into a temporary cache tree independent of the reconstruction process; After the switching step is completed, all interest points in the temporary cache tree are merged and inserted into the newly constructed balanced KD tree; The query service copy and temporary cache tree are destroyed, and the data insertion and query services are taken over by a newly balanced KD tree.

8. A device for constructing a dynamically balanced KD-tree index based on real-time streaming data, characterized in that, The device includes: A target trajectory flow determination module is used to determine a target trajectory flow, wherein the target trajectory flow includes multiple points of interest arranged based on time series points; The initial spatial range determination module is used to determine the initial spatial range of the root node of the initial KD tree based on the geographical range of the target trajectory flow. The initial child node creation module is used to divide the initial spatial range of the root node into two initial subspaces using the coordinates of the first point of interest as the dividing point, and to create an initial child node corresponding to each initial subspace. The dynamic insertion operation execution module is used to perform a dynamic insertion operation for each target interest point reached after the first interest point: in the current structure of the initial KD tree, determine the corresponding leaf node for each target interest point and store the target interest point in the corresponding leaf node; The split operation triggering module is used to trigger a split operation for any leaf node when the number of points of interest stored in any leaf node exceeds a first preset capacity threshold, converting it into an internal node and creating two new leaf nodes based on its median splitting rule. The global reconstruction process triggering module is used to monitor the comprehensive imbalance index of the initial KD tree in real time. When the comprehensive imbalance index value exceeds the preset reconstruction threshold, the global reconstruction process is triggered. The global reconstruction process reconstructs a new balanced KD tree based on the interest points stored in all leaf nodes of the initial KD tree up to the trigger time.

9. An electronic device, characterized in that, The electronic device includes a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to cause the electronic device to perform the steps of the construction method of the dynamic balancing KD-tree index based on real-time streaming data as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the various steps of the method for constructing a dynamically balanced KD-tree index based on real-time streaming data as described in any one of claims 1-7.

Images