Space-air-ground multi-terminal intelligent data fusion processing system and method

Abstract

The application provides a space-air-ground multi-terminal intelligent data fusion processing system and method, including unified space-based and ground-based data coordinate systems, extraction of point space-time and sensing values to construct a multi-dimensional feature vector; construction of an index tree, node splitting storage of a covariance matrix and an inverse matrix; when a feature value ratio is higher than a structure discrimination threshold which monotonically increases with node depth and is constrained by a preset upper limit, directional capsule body surrounding is adopted, otherwise minimum enclosing sphere is adopted; input of an air query point, local Mahalanobis distance is solved by means of the covariance inverse matrix stored by the node and is converted into a global normalized distance scalar; the nearest neighbor of the last moment is initialized as a priority search area, and nodes in the area are preferentially enqueued; if the query point is more than the current optimal neighbor distance from the candidate enclosing body boundary, direct pruning is performed; the suboptimal node closest to the query point is recorded, if the approximate nearest neighbor distance is greater than the node boundary distance, the nearest neighbor is searched and updated, and a data record of the query point and the nearest neighbor is generated.

Description

Technical Field

[0001] This application belongs to the field of data fusion processing, and in particular relates to an intelligent data fusion processing system and method for multiple terminals in air, space, and ground. Background Technology

[0002] Inspection drones play a crucial role in tasks such as power line inspection, oil and gas pipeline monitoring, natural disaster assessment, and wide-area infrastructure inspection. However, limitations in flight altitude, endurance, and sensor field of view mean that data collected from a single aerial perspective is insufficient to meet the demands of large-scale, comprehensive, and long-term three-dimensional monitoring. By integrating space-based remote sensing data, ground-based IoT sensor data, and airborne data collected by inspection drones, an integrated air-space-ground data fusion system can be constructed. This system organically combines satellite macroscopic observation capabilities, ground-based node local monitoring capabilities, and drone maneuvering close-range reconnaissance capabilities, overcoming the information acquisition bottleneck of single-source sensors and forming a multi-dimensional, panoramic geographic information view. However, space-based and ground-based data are typically massive in volume and simultaneously contain spatial location, temporal series, and multiple physical attributes, forming complex high-dimensional datasets. When drones perform inspection missions, the real-time aerial data needs to quickly find the best correlation objects from the high-dimensional background data to achieve spatiotemporal registration and fusion of cross-source data. Existing spatial data indexing structures and nearest-neighbor search mechanisms struggle to adapt to the internal topology of high-dimensional, non-uniformly distributed multi-source datasets, easily leading to overlapping region searches and redundant calculations, thus reducing matching efficiency. In continuous UAV flight scenarios, limited onboard computing resources and high real-time requirements present significant challenges. How to construct an organizational structure that conforms to the true distribution patterns of multi-source data while ensuring data association accuracy, and how to design search strategies that can quickly eliminate redundant regions and reduce global exhaustive calculations, thereby achieving real-time fusion and record generation of air-space-ground data, has become a pressing engineering challenge in this field. Summary of the Invention

[0003] This invention provides an intelligent data fusion processing system and method for multiple terminals in air, space, and ground.

[0004] According to one aspect of the present invention, a multi-terminal intelligent data fusion processing system for air, space, and ground is provided, the system comprising: The module is used to unify space-based and ground-based data into the same geographic coordinate system, extract the spatial location, timestamp, and sensor physical quantities of data points, and construct a multi-dimensional feature vector set. The calculation module is used to construct a data index tree based on the multi-dimensional feature vector set. During the node splitting process, it calculates and stores the covariance matrix and inverse matrix of the data point set within the node. When the ratio of the largest eigenvalue to the second largest eigenvalue of the covariance matrix is ​​greater than the structure discrimination threshold, the structure discrimination threshold increases monotonically with the node depth and is constrained by a preset upper limit threshold. The oriented capsule with the first principal component vector as the central axis is used as the bounding body of the node; otherwise, the minimum bounding sphere is used as the bounding body. The generation module receives the current air data point as the query point, calculates the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate node in the tree, and converts it into a global distance. It initializes the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, places the nodes in the region at the front of the queue to be queried, and removes the candidate node if the distance between the query point and the boundary of the bounding body of the candidate node is greater than the optimal nearest neighbor distance. It records the node in the queue whose boundary of the bounding body is closest to the query point as the second-best candidate node. If the approximate nearest neighbor distance is greater than the boundary distance of the second-best candidate node, it performs a local search to update the nearest neighbor of the second-best candidate node, and generates data records based on the association between the nearest neighbor and the query point.

[0005] According to another aspect of the present invention, a method for intelligent data fusion processing across multiple terminals in airspace, space, and ground is provided, the method comprising: Unify space-based and ground-based data into the same geographic coordinate system, extract the spatial location, timestamp, and sensor physical quantities of data points, and construct a multi-dimensional feature vector set; A data index tree is constructed based on the multi-dimensional feature vector set. During the node splitting process, the covariance matrix and inverse matrix of the data point set within the node are calculated and stored. When the ratio of the largest eigenvalue to the second largest eigenvalue of the covariance matrix is ​​greater than the structure discrimination threshold, the structure discrimination threshold increases monotonically with the node depth and is constrained by a preset upper limit threshold. An oriented capsule with the first principal component vector as the central axis is used as the bounding body of the node; otherwise, the minimum bounding sphere is used as the bounding body. The system receives the current air data point as the query point, calculates the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate node in the tree, and converts it into a global distance. It initializes the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, places the nodes in the region at the front of the queue to be queried, and removes the candidate node if the distance between the query point and the boundary of the bounding body of the candidate node is greater than the optimal nearest neighbor distance. It records the node in the queue whose boundary of the bounding body is closest to the query point as the second-best candidate node. If the approximate nearest neighbor distance is greater than the boundary distance of the second-best candidate node, it performs a local search to update the nearest neighbor of the second-best candidate node, and generates data records based on the association between the nearest neighbor and the query point.

[0006] This invention provides a standardized foundation for data fusion by unifying the coordinates of multi-source heterogeneous data and extracting multi-dimensional features. When constructing the index tree, a structure discrimination threshold that monotonically increases with node depth and is constrained by a preset upper limit threshold is used in conjunction with the ratio of the covariance matrix eigenvalues. This allows for the selection of oriented capsules or minimum bounding spheres for the geometric distribution of the data, reducing redundant blank areas within the bounding spheres. In the query matching stage, a pre-stored covariance inverse matrix is ​​reused to calculate Mahalanobis distance, combining the correlation between features to improve the accuracy of distance measurement. The spatiotemporal continuity of UAV inspection trajectories is utilized to guide priority searches using the nearest neighbor points from the previous time step, reducing the range of invalid traversals and improving retrieval speed. In the pruning determination, the distances to candidate bounding sphere boundaries, the current best nearest neighbor distance, and the distances to the second-best candidate node boundaries are compared under a unified normalized feature distance scale. A second-best candidate node monitoring mechanism intervenes in local searches when there is a risk of deviation in approximate searches, balancing execution efficiency and data association accuracy in large-scale air-space-ground data fusion. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the oriented capsule enclosure structure based on the first principal component; Figure 2 A comparison of the nearest-distance convergence performance of the two search initialization mechanisms. Detailed Implementation

[0008] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0009] This application discloses an intelligent data fusion processing system for multiple terminals across air, space, and ground, comprising: The module is used to unify space-based and ground-based data into the same geographic coordinate system, extract the spatial location, timestamp, and sensor physical quantities of data points, and construct a multi-dimensional feature vector set.

[0010] The coordinate transformation function in the open-source spatial projection library GDAL is called, and a seven-parameter transformation algorithm is used to uniformly transform the space-based satellite imagery data in the WGS84 geographic coordinate system and the ground-based sensor radar data in the CGCS2000 coordinate system to the UTM projection coordinate system. The east distance coordinate, north distance coordinate, and elevation coordinate corresponding to all data points are extracted as the three-dimensional spatial position. If the WGS84 geographic coordinate system is used as the unified calculation coordinate system, longitude, latitude, and elevation are extracted as the three-dimensional spatial position. The above two coordinate representation methods are used only in the same implementation process and are not mixed in the same distance calculation. When the UTM projection coordinate system is used, a unified UTM zone number is pre-determined according to the inspection operation area, and all space-based, ground-based, and aerial data are uniformly transformed to the projection coordinates under that zone number. If the operation area spans multiple UTM zones, a unified geocentric coordinate system or a local engineering coordinate system is used instead of the UTM projection coordinate system for distance calculation.

[0011] The time text information carried by each data item is parsed, and the time processing module is called to convert it into a unified UNIX timestamp value.

[0012] Three physical quantities—temperature, relative humidity, and carbon dioxide concentration—monitored by the sensors were extracted, and the physical quantity data were normalized to the range of 0 to 1.

[0013] The extracted spatial locations, timestamps, and physical quantities are horizontally concatenated along the feature dimension to generate a set of multi-dimensional feature vectors in a unified format, which are then stored in a memory array.

[0014] In some embodiments, the extraction of spatial location, timestamp, and sensor physical quantities of data points to construct a multi-dimensional feature vector set includes: Extract the three-dimensional spatial coordinates of space-based and ground-based data points in a unified coordinate system as their three-dimensional spatial positions; Convert the data collection time into a numerical timestamp scalar; Temperature, relative humidity, and carbon dioxide concentration values ​​of fixed dimensions are extracted as physical quantities of the sensor. The three-dimensional spatial position, timestamp scalar, and sensor physical quantity are respectively processed by dimensionless normalization and then concatenated in order to construct a multi-dimensional floating-point array as a feature vector.

[0015] In this embodiment, by parsing space-based and ground-based data packets, the original latitude and longitude coordinates are uniformly projected to the UTM coordinate system to obtain the east distance coordinates, north distance coordinates, and elevation values. In another optional embodiment, the longitude, latitude, and elevation values ​​can also be directly used as the three-dimensional spatial location in the WGS84 coordinate system. To ensure consistency in distance calculation, only one spatial coordinate expression method is used within the same index tree.

[0016] The standard acquisition time of the data is converted into a 64-bit integer Unix millisecond timestamp scalar, for example, 12:00 on October 1, 2023 is converted to 1696132800000; at the same time, fixed 3-dimensional physical quantities, including temperature, relative humidity and carbon dioxide concentration, are extracted from the sensor readings.

[0017] After extracting the raw data, the Min-Max normalization algorithm was used to perform dimensionless processing on the data with different dimensions. By using preset historical empirical extreme values, the three-dimensional spatial position, timestamp scalar, and 3D sensor physical quantities were uniformly mapped to the standard floating-point interval [0,1].

[0018] Following the order of east distance coordinates, north distance coordinates, elevation, timestamp, temperature, humidity, and carbon dioxide concentration, the seven normalized values ​​are concatenated continuously in memory to construct a single-precision or double-precision floating-point array of length 7. This array serves as the multidimensional feature vector representation for the data point in spatial calculations. In an optional embodiment using WGS84 geographic coordinates, the aforementioned east distance coordinates and north distance coordinates are replaced with longitude and latitude, respectively.

[0019] The calculation module is used to construct a data index tree based on the multi-dimensional feature vector set. During the node splitting process, it calculates and stores the covariance matrix and inverse matrix of the data point set within the node. When the ratio of the largest eigenvalue to the second largest eigenvalue of the covariance matrix is ​​greater than the structure discrimination threshold, the structure discrimination threshold increases monotonically with the node depth and is constrained by a preset upper limit threshold. The oriented capsule with the first principal component vector as the central axis is used as the bounding body of the node; otherwise, the minimum bounding sphere is used as the bounding body.

[0020] Initialize the root node object of the tree structure and store the set of multi-dimensional feature vectors from the memory array into the root node. Divide the dataset of the current node into two subsets based on spatial feature distance, thereby generating left and right child nodes.

[0021] Calculate the covariance matrix of all multidimensional eigenvectors within the current child node, calculate the inverse of this covariance matrix, and store it as a private attribute of the node class in the tree structure object. Call the `eig` eigenvalue decomposition function to solve the covariance matrix, extract all eigenvalues, and sort them in descending order to obtain the largest and second-largest eigenvalues. When calculating the ratio of the largest to the second-largest eigenvalue, use a regularized covariance matrix for eigenvalue decomposition, and take the larger of the second-largest eigenvalue and a preset minimum positive number ε as the denominator. ε is a floating-point constant greater than 0 to avoid abnormally amplified ratios when the second-largest eigenvalue is zero or close to zero.

[0022] Extract the depth value of the current node in the tree structure, multiply the preset basic structure discrimination threshold by the node depth raised to the power of an increasing base greater than 1, and take the smaller value between the preset upper limit threshold and the lower limit threshold to calculate the structure discrimination threshold for the current depth. This ensures that the threshold increases monotonically with the node depth and is subject to the upper limit constraint.

[0023] The ratio of the largest eigenvalue to the second largest eigenvalue is compared to see if it exceeds the structure discrimination threshold. If it does, the feature vector extraction function is called to obtain the eigenvector corresponding to the largest eigenvalue as the first principal component vector. Using this first principal component vector as the central axis direction, the range of projected coordinates of all multidimensional data points within the node along the central axis direction is calculated. Using the node mean vector as the reference point, the vectors are extended along the first principal component vector direction to the positions corresponding to the minimum and maximum projected scalar values, respectively, to determine the starting and ending points of the central line segment inside the capsule. The Mahalanobis orthogonal distances from all data points to the central axis segment are calculated, and the maximum value is extracted as the cross-sectional radius of the oriented capsule. Various spatial parameters of the oriented capsule are generated and bound to the current node attributes.

[0024] If the value is not greater than the structure discrimination threshold, the Welzl stochastic incremental algorithm is invoked to solve for the minimum bounding sphere of the data point set within the current node. The coordinates of the sphere's center and radius in multidimensional space are calculated, and the sphere parameters are generated as the bounding volume of the node. The center and radius of the minimum bounding sphere are calculated based on the same normalized feature space and are used for subsequent comparison with the boundary distance after unification with the Mahalanobis distance scale.

[0025] Using a recursive approach, repeat the above process of splitting nodes into KMeans and generating bounding volumes until the number of data points in the newly generated child nodes is less than the preset upper limit of the leaf node capacity, thus completing the construction of the entire data index tree.

[0026] In some embodiments, the covariance matrix and inverse matrix of the data point set within the computing node are stored together, including: Calculate the multidimensional mean vector of the feature vectors of all data points within the current node; Subtract the multidimensional mean vector from the feature vector of each data point to obtain the offset vector of each data point; Calculate the sum of the products of all offset vectors and the transpose matrix, and divide by the total number of data points within the node to generate the standard covariance matrix; Add positive constants to each element of the main diagonal of the standard covariance matrix to generate a non-singular positive definite matrix; The positive definite matrix is ​​inverted using the Gauss-Jordan elimination algorithm to obtain a two-dimensional floating-point inverse matrix, which is then cached and recorded as a unique attribute variable for the current node.

[0027] For the constructed feature vector set, such as a feature dimension of 7, traverse the N data points in the current node, such as N=1000, sum by column and divide by the total number N to calculate the 7-dimensional mean vector of the local data distribution in the node.

[0028] For each data point, the offset vector is calculated by subtracting the mean vector from the 7×1 single-point feature column vector to obtain the difference in direction and magnitude. Then, using matrix multiplication, the 7×1 offset vector of each data point is multiplied by its 1×7 transpose to obtain a 7×7 local scatter matrix. The sum of all N local scatter matrices is then divided by the total number of data points within the node, N, to generate a 7×7 standard covariance matrix representing the correlation between each feature dimension.

[0029] Add a preset regularization positive constant to each element of the main diagonal of the standard covariance matrix, for example... This generates a non-singular positive definite matrix. Then, the Gauss-Jordan elimination method is used to perform elementary row operations on the 7×7 matrix to invert it, and its associated two-dimensional floating-point inverse matrix is ​​calculated.

[0030] The inverse matrix is ​​persistently cached as a two-dimensional array of double-precision floating-point numbers and registered as a dedicated attribute variable in the memory data structure of the current node object.

[0031] In some embodiments, the structure discrimination threshold monotonically increases with increasing node depth and is constrained by a preset upper limit threshold, including: Set the node depth of the root node of the index tree to 0, and set the initial structure discrimination threshold constant and a preset upper limit threshold greater than the initial structure discrimination threshold constant; Set a base that is greater than 1 and increases progressively at each level; During node splitting, the node depth integer D of the current target node is obtained, the initial structure discrimination threshold is multiplied by the D power of the increasing base, and the minimum value is taken with the preset upper limit threshold to calculate and generate a single-precision floating-point value, which is used as the structure discrimination threshold of the current node.

[0032] During the initialization phase of building the data index tree, the depth parameter of the root node at the top level of the tree structure is initialized to an integer 0, and an initial structure discrimination threshold constant is configured for it based on prior experience. The preferred range of this constant can be set between 10.0 and 50.0. For example, setting it to 15.0 means that within the broad space where the root node is located, a directional capsule is used as the enclosing body only when the data distribution shows significant anisotropy.

[0033] Meanwhile, a progressively increasing base value greater than 1 is set in the configuration table, preferably between 1.05 and 1.20, for example, 1.15, to control the rate at which the discrimination threshold increases exponentially with the depth of the layer; and a preset upper limit threshold is set, for example, 50.0, to avoid the threshold of deep nodes from increasing excessively.

[0034] During the recursive execution of index tree node splitting, the computing engine extracts the integer D corresponding to the level depth of the current target node in real time. For example, if the third level node is currently being processed, D=3.

[0035] The power function of the math library is called to multiply the initial threshold of 15.0 by the cube of the increment base of 1.15, and the increment ratio of 1.520875 is calculated. Thus, the structure discrimination threshold of the current node is approximately 22.813125. If the calculated value exceeds the preset upper limit threshold, the preset upper limit threshold is taken as the structure discrimination threshold of the current node.

[0036] The calculation result is converted into a 32-bit single-precision floating-point number and assigned to the current node. This ensures that as the tree depth increases, the criteria for determining the shape of the bounding volume become more stringent, causing smaller bounding volumes at the bottom layer to tend to choose the minimum bounding sphere form with lower computational overhead.

[0037] In some embodiments, using an oriented capsule with the first principal component vector as its central axis as the bounding volume of the node includes: Singular value decomposition is performed on the covariance matrix, and the unit vector corresponding to the largest eigenvalue is extracted as the first principal component vector; Calculate the scalar value of the projection of the feature vector of each data point within the node onto the multidimensional spatial axis where the first principal component vector is located; Obtain the projection points corresponding to the maximum and minimum projection values, and take the closed spatial line segment between the two extreme projection points as the central axis segment of the directional capsule. The orthogonal distance from the eigenvector of each data point within a node to the central axis segment is calculated by combining the inverse matrix of the covariance matrix. The largest orthogonal distance is selected as the multidimensional unified surface boundary radius of the directional capsule, and the directional capsule is generated.

[0038] The SVD singular value decomposition algorithm is called to perform dimensionality reduction decomposition on the 7×7 covariance matrix of the node cache, extract the first principal eigenvalue with the largest value, for example, eigenvalue 3.25, and its corresponding 7-dimensional eigenvector, and then perform L2 norm normalization on the vector to generate a unit vector, which is used as the first principal component vector representing the narrow direction of the data cluster distribution.

[0039] Traverse all the original data points contained in the node, using the node mean vector as the projection reference, and perform a dot product operation between the offset vector of each data point relative to the node mean vector and the first principal component vector to calculate the projection scalar value of the feature vector of each data point on the axis of the first principal component vector. For example, the calculated projection values ​​of all points are distributed in the range of -2.4 to 5.8.

[0040] Based on the calculated projection values, the minimum projection value of -2.4 and the maximum projection value of 5.8 are selected. The node mean vector is then shifted along the direction of the first principal component vector by the lengths corresponding to -2.4 and 5.8, respectively, to obtain two axis endpoints located on the same first principal component axis. Connecting these two axis endpoints yields the central axis segment of the oriented capsule. The original data points corresponding to the extreme projection values ​​are only used to determine the projection range and are not directly used as the central axis by the line connecting the two original data points.

[0041] Calculate the shortest projected perpendicular vector from the feature vector of each data point within the node to the central axis segment, and combine it with the previously cached covariance inverse matrix to calculate the Mahalanobis norm of the perpendicular vector, that is, the Mahalanobis orthogonal distance from each point to the axis.

[0042] The maximum value is selected from the above distance set, for example, the orthogonal distance is 1.15. This value is set as the radius of the unified curved surface boundary of the two hemispheres and the cylindrical body of the directional capsule, generating a directional capsule structure with multidimensional volume and orientation. The morphology of the directional capsule in the two-dimensional principal component projection space is as follows: Figure 1 As shown, the spatial correspondence between the data cluster distribution direction, the central axis of the first principal component, and the boundary enclosing the capsule is illustrated. In this embodiment, the radius of the oriented capsule is uniformly determined using the Mahalanobis orthogonal distance, and is no longer mixed with the Euclidean perpendicular distance.

[0043] The generation module receives the current air data point as the query point, calculates the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate node in the tree, and converts it into a global distance. It initializes the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, places the nodes in the region at the front of the queue to be queried, and removes the candidate node if the distance between the query point and the boundary of the bounding body of the candidate node is greater than the optimal nearest neighbor distance. It records the node in the queue whose boundary of the bounding body is closest to the query point as the second-best candidate node. If the approximate nearest neighbor distance is greater than the boundary distance of the second-best candidate node, it performs a local search to update the nearest neighbor of the second-best candidate node, and generates data records based on the association between the nearest neighbor and the query point.

[0044] The robot operating system (ROS) receives real-time multi-dimensional perception features from the onboard equipment of the inspection drone and encapsulates them into query point vectors. It traverses candidate nodes downwards from the root node of the data index tree, calls the spatial distance module, and calculates the Mahalanobis distance between the query point vector and the center coordinate vector of the candidate node using the inverse matrix of the pre-stored covariance matrix of the candidate nodes. Based on a pre-calibrated dimensionless distance calibration coefficient, the local adaptive Mahalanobis distance is converted into a globally normalized distance scalar that is comparable across the entire tree. This globally normalized distance scalar is used for sorting, pruning, and nearest neighbor updates, and is not equivalent to metric Euclidean distance. When the actual geospatial distance needs to be output, a metric Euclidean distance is calculated separately based on the UTM spatial coordinate sub-vector; this metric Euclidean distance is not used in tree search pruning decisions.

[0045] The system reads the nearest neighbor identifier stored in the system's Redis cache database from the previous time frame query task. It then traverses backwards through the tree structure to locate the leaf node containing the nearest neighbor from the previous time frame and its adjacent parent node levels. The spatial region covered by these associated nodes is designated as the priority search region. The system then calls Python's built-in PriorityQueue data structure, setting the priority weight of nodes within the priority search region to the lowest value, thus placing them at the head of the query queue for priority dequeueing and branch traversal.

[0046] For each candidate node leaving the queue, the normalized boundary distance from the query point to the bounding body boundary is calculated based on the type of bounding body it is bound to. If it is a minimum bounding sphere, the distance from the query point to the center of the sphere is calculated in the normalized feature space and the sphere radius is subtracted. If it is a directional capsule, the orthogonal distance from the query point to the internal central axis segment is calculated at the same local Mahalanobis scale and the cross-sectional radius is subtracted. The bounding body boundary distance obtained in this way is at the same normalized distance scale as the optimal nearest neighbor distance and can be directly used for pruning judgment.

[0047] Determine if the boundary distance is numerically greater than the current value of the best nearest neighbor distance variable maintained in memory throughout the entire search process. The boundary distance, best nearest neighbor distance, and second-best candidate node boundary distance are all represented using the same normalized feature distance scale. If it is greater, a pruning algorithm is used on the tree search path to terminate branch access to the candidate node and all its subsequent subtrees, achieving pruning. Simultaneously, in the remaining priority queue of queries, find the node instance with the smallest calculated bounding volume boundary distance to the query point, record its object reference pointer as a second-best candidate node, and continue the search down the hierarchy.

[0048] Upon reaching a leaf node in the tree, the system iterates through the nodes, calculating the local adaptive Mahalanobis distance between the query point and each data point within the leaf node. This distance is then converted to a globally normalized distance scalar. The point with the closest normalized distance is identified as the approximate nearest neighbor, and its corresponding distance value is recorded as the approximate nearest neighbor distance. If a conditional check indicates that this approximate nearest neighbor distance is numerically greater than the bounding volume boundary distance of the second-best candidate node just recorded by the system, it suggests the possibility of a closer target data point within the spatial range corresponding to the second-best candidate node. In this case, a local search mechanism is triggered. After triggering the local search, the system repeatedly extracts the candidate node with the smallest bounding volume boundary distance from the unvisited candidate node set and performs a local search, updating the current optimal nearest neighbor distance after each local search. The compensation search process terminates when the bounding volume boundary distances of all unvisited candidate nodes are not less than the current optimal nearest neighbor distance, or when the preset search count limit is reached.

[0049] For each extracted current compensation candidate node, the Depth-First Search (DFS) algorithm is called to recursively traverse all its underlying branches. The local adaptive Mahalanobis distance and the corresponding global normalized distance scalar between each specific data point in the current compensation candidate node and the query point are calculated one by one. If there is a data point in the current compensation candidate node with a distance smaller than the current best nearest neighbor, then the data point is updated as the current best nearest neighbor, and the current best nearest neighbor distance is updated synchronously.

[0050] The feature vector of the current inspection drone's query point is received and merged with the ground-based or space-based feature vector of the current best nearest neighbor point found by the system. The same attribute field is concatenated to generate a two-dimensional structured data record that contains both the fusion timestamp and the integrated air-space-ground monitoring attributes. The record is then stored in the fusion data table of the MySQL relational database using an SQL insert statement.

[0051] In some embodiments, calculating the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate nodes of the tree includes: Load the inverse matrix M stored in the attributes of the current candidate node or the second-best candidate node; Obtain the feature vector Q of the query point and the feature vector P of the test point, calculate the feature difference at the corresponding index position, and generate the difference vector. The transpose of the difference vector, the inverse matrix M, and the difference vector are multiplied sequentially to obtain the scalar sum of the products; The square root of the sum of the scalars is taken to obtain the local adaptive Mahalanobis distance between the query point and the test point.

[0052] When the nearest neighbor search algorithm triggers the calculation of spatial distance, the calculation engine reads and loads the previously constructed and stored 7×7 two-dimensional floating-point inverse matrix M from the memory block attributes of the candidate node currently being traversed or the node that has been marked as a suboptimal candidate node.

[0053] Obtain the 7-dimensional query point feature vector Q currently collected by the drone, such as [0.5,0.2,0.1,0.8,0.4,0.6,0.3], and the test point feature vector P currently participating in the comparison in the tree structure, such as [0.4,0.2,0.2,0.7,0.4,0.5,0.3]. Calculate the difference of each feature dimension by performing a positional subtraction operation between the two vectors, thereby generating a 7×1 columnar difference vector. The calculation result is the transpose vector of [0.1,0.0,-0.1,0.1,0.0,0.1,0.0].

[0054] After obtaining the difference vector, matrix multiplication is performed sequentially based on the quadratic Mahalanobis distance formula. The transpose of the 7×1 difference vector is calculated, which is transformed into a 1×7 row vector. The product of this transpose and the loaded 7×7 inverse matrix M yields a 1×7 intermediate row vector.

[0055] Multiply the intermediate row vector by the inner product of the original 7×1 difference vector, and sum the variance-scaled projection differences across all dimensions to obtain the scalar sum representing the squared distance value. For example, the summation result is 0.144.

[0056] The floating-point arithmetic unit is invoked to perform a square root operation on the sum of the scalars, yielding 0.379. This results in a locally adaptive Mahalanobis distance, which removes the dimensional differences of multidimensional heterogeneous features and adapts to the directional distribution characteristics of local data clusters. When used for cross-node sorting and pruning, this locally adaptive Mahalanobis distance is further converted into a globally normalized distance scalar using a dimensionless calibration coefficient; this calibration process does not involve directly multiplying the Mahalanobis distance by the total variance.

[0057] In some embodiments, the step of initializing the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, and placing the nodes within the region at the front of the query queue, includes: Get the nearest neighbor leaf node number confirmed after the query point completed the search at the previous time step; In the tree structure, trace upwards two levels to find the grandparent node of the nearest neighbor leaf node, and divide the space under the bounding volume of the grandparent node into a priority search region. Read the sequence of all child nodes and all leaf nodes under the grandfather node, and insert all nodes in the sequence into the front of the system's query queue according to the storage order, as the first batch of test nodes.

[0058] Because the data query requests between two adjacent sampling times have high spatiotemporal continuity and location similarity during continuous flight missions of inspection drones, the global unique identifier of the leaf node to which the nearest neighbor data point belongs, such as NodeID:1054, is read and parsed from the global search cache of the previous sampling period.

[0059] Based on the parent node tracing pointer integrated inside the index tree node, two consecutive tracing operations are performed upwards along the tree topology starting from the leaf node to locate the grandparent level node of the leaf node, such as NodeID:263. The multidimensional space volume covered by the enclosing volume of the grandparent node is then divided and identified as the priority search area that is likely to exist at the current moment.

[0060] After defining the priority region, the subtree branches belonging to the grandparent node are traversed, recursively reading all its internal child nodes and all leaf nodes storing data at the bottom, forming a flattened spatial node sequence set. A priority queue mechanism is used as the underlying data structure of the global query queue. The priority weight of nodes within the priority search region is set to a value lower than that of ordinary nodes, and they are enqueued sequentially according to their original storage order, allowing them to be dequeued first and participate in the search.

[0061] This initialization mechanism prioritizes comparing the first batch of test nodes with high spatiotemporal correlation before starting the regular traversal. This optimizes the initial value of the optimal nearest neighbor distance generated in the first round, thereby improving the removal efficiency of global space pruning. A comparison of the nearest neighbor convergence processes of the two search initialization mechanisms is provided below. Figure 2 As shown, the grandfather node priority region initialization strategy can quickly converge to a better nearest neighbor distance with less candidate node traversal.

[0062] In some embodiments, performing a local search to update the nearest neighbor for the suboptimal candidate node includes: Extract the recorded suboptimal candidate nodes, or the set of all underlying raw data points contained in the current compensation candidate node extracted from the unvisited candidate node set; Traverse the underlying set of original data points and calculate the local adaptive Mahalanobis distance between the query point and each original data point; Find the minimum value among the calculated local adaptive Mahalanobis distances. If the minimum value is less than the current best nearest neighbor distance, then take the original data point corresponding to the minimum value as the updated current best nearest neighbor point, and update the current best nearest neighbor distance simultaneously.

[0063] In the process of approximate search based on bounding volume distance, if the threshold judgment of the search condition is triggered, the underlying storage block associated with the suboptimal candidate node will be located and accessed according to the pointer of the suboptimal candidate node previously recorded and saved in the search queue. All the underlying original data points encapsulated and maintained by the suboptimal node will be extracted. Typically, each leaf node encapsulates dozens to hundreds of 7-dimensional multi-dimensional feature points. For example, 50 original space-based or ground-based fusion data points contained in the target leaf node can be extracted.

[0064] Initiate an iterative process to apply the aforementioned formula combining the local covariance inverse matrix to each of the 50 extracted raw data points, calculate the actual local adaptive Mahalanobis distance between the current UAV query point and each raw data point in the set, and convert it into a globally normalized distance scalar according to the same dimensionless calibration rule as the main search phase.

[0065] During the traversal calculation of the original data points, a local minimum temporary variable is updated and maintained to record the shortest feature distance found within the current suboptimal node range and its corresponding point index. After all points within the node have been compared, the minimum Mahalanobis distance within this local range is extracted; assuming the calculated minimum value is 0.28.

[0066] At this point, the calculated minimum value of 0.28 is compared with the current best nearest neighbor distance, assumed to be 0.32. Since 0.28 is less than 0.32, the original data point corresponding to the local distance of 0.28 is updated to the current best nearest neighbor point, and the current best nearest neighbor distance is updated simultaneously. This compensatory verification step achieves a closed loop of fusion between approximate fast pruning and local correction by filling in gaps at high-probability overlapping edges. The above comparison process is always completed under the same normalized distance scale, avoiding direct mixing of Euclidean distance, Mahalanobis distance, and physical metric distance.

[0067] This application discloses a method for intelligent data fusion processing across multiple terminals in air, space, and ground, the method comprising: Unify space-based and ground-based data into the same geographic coordinate system, extract the spatial location, timestamp, and sensor physical quantities of data points, and construct a multi-dimensional feature vector set; A data index tree is constructed based on the multi-dimensional feature vector set. During the node splitting process, the covariance matrix and inverse matrix of the data point set within the node are calculated and stored. When the ratio of the largest eigenvalue to the second largest eigenvalue of the covariance matrix is ​​greater than the structure discrimination threshold, the structure discrimination threshold increases monotonically with the node depth and is constrained by a preset upper limit threshold. An oriented capsule with the first principal component vector as the central axis is used as the bounding body of the node; otherwise, the minimum bounding sphere is used as the bounding body. The system receives the current air data point as the query point, calculates the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate node in the tree, and converts it into a global distance. It initializes the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, places the nodes in the region at the front of the queue to be queried, and removes the candidate node if the distance between the query point and the boundary of the bounding body of the candidate node is greater than the optimal nearest neighbor distance. It records the node in the queue whose boundary of the bounding body is closest to the query point as the second-best candidate node. If the approximate nearest neighbor distance is greater than the boundary distance of the second-best candidate node, it performs a local search to update the nearest neighbor of the second-best candidate node, and generates data records based on the association between the nearest neighbor and the query point.

[0068] In some embodiments, the extraction of spatial location, timestamp, and sensor physical quantities of data points to construct a multi-dimensional feature vector set includes: Extract the three-dimensional spatial coordinates of space-based and ground-based data points in a unified coordinate system as their three-dimensional spatial positions; Convert the data collection time into a numerical timestamp scalar; Temperature, relative humidity, and carbon dioxide concentration values ​​of fixed dimensions are extracted as physical quantities of the sensor. The three-dimensional spatial position, timestamp scalar, and sensor physical quantity are respectively processed by dimensionless normalization and then concatenated in order to construct a multi-dimensional floating-point array as a feature vector.

[0069] In some embodiments, the covariance matrix and inverse matrix of the data point set within the computing node are stored together, including: Calculate the multidimensional mean vector of the feature vectors of all data points within the current node; Subtract the multidimensional mean vector from the feature vector of each data point to obtain the offset vector of each data point; Calculate the sum of the products of all offset vectors and the transpose matrix, and divide by the total number of data points within the node to generate the standard covariance matrix; Add positive constants to each element of the main diagonal of the standard covariance matrix to generate a non-singular positive definite matrix; The positive definite matrix is ​​inverted using the Gauss-Jordan elimination algorithm to obtain a two-dimensional floating-point inverse matrix, which is then cached and recorded as a unique attribute variable for the current node.

[0070] In some embodiments, the structure discrimination threshold monotonically increases with increasing node depth and is constrained by a preset upper limit threshold, including: Set the node depth of the root node of the index tree to 0, and set the initial structure discrimination threshold constant and a preset upper limit threshold greater than the initial structure discrimination threshold constant; Set a base that is greater than 1 and increases progressively at each level; During node splitting, the node depth integer D of the current target node is obtained, the initial structure discrimination threshold is multiplied by the D power of the increasing base, and the minimum value is taken with the preset upper limit threshold to calculate and generate a single-precision floating-point value, which is used as the structure discrimination threshold of the current node.

[0071] In some embodiments, using an oriented capsule with the first principal component vector as its central axis as the bounding volume of the node includes: Singular value decomposition is performed on the covariance matrix, and the unit vector corresponding to the largest eigenvalue is extracted as the first principal component vector; Calculate the scalar value of the projection of the feature vector of each data point within the node onto the multidimensional spatial axis where the first principal component vector is located; Obtain the projection points corresponding to the maximum and minimum projection values, and take the closed spatial line segment between the two extreme projection points as the central axis segment of the directional capsule. The orthogonal distance from the eigenvector of each data point within a node to the central axis segment is calculated by combining the inverse matrix of the covariance matrix. The largest orthogonal distance is selected as the multidimensional unified surface boundary radius of the directional capsule, and the directional capsule is generated.

[0072] In some embodiments, calculating the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate nodes of the tree includes: Load the inverse matrix M stored in the attributes of the current candidate node or the second-best candidate node; Obtain the feature vector Q of the query point and the feature vector P of the test point, calculate the feature difference at the corresponding index position, and generate the difference vector. The transpose of the difference vector, the inverse matrix M, and the difference vector are multiplied sequentially to obtain the scalar sum of the products; The square root of the sum of the scalars is taken to obtain the local adaptive Mahalanobis distance between the query point and the test point.

[0073] In some embodiments, the step of initializing the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, and placing the nodes within the region at the front of the query queue, includes: Get the nearest neighbor leaf node number confirmed after the query point completed the search at the previous time step; In the tree structure, trace upwards two levels to find the grandparent node of the nearest neighbor leaf node, and divide the space under the bounding volume of the grandparent node into a priority search region. Read the sequence of all child nodes and all leaf nodes under the grandfather node, and insert all nodes in the sequence into the front of the system's query queue according to the storage order, as the first batch of test nodes.

[0074] In some embodiments, performing a local search to update the nearest neighbor for the suboptimal candidate node includes: Extract the recorded suboptimal candidate nodes, or the set of all underlying raw data points contained in the current compensation candidate node extracted from the unvisited candidate node set; Traverse the underlying set of original data points and calculate the local adaptive Mahalanobis distance between the query point and each original data point; Find the minimum value among the calculated local adaptive Mahalanobis distances. If the minimum value is less than the current best nearest neighbor distance, then take the original data point corresponding to the minimum value as the updated current best nearest neighbor point, and update the current best nearest neighbor distance simultaneously.

[0075] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0076] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A multi-terminal intelligent data fusion processing system for air, space, and ground, characterized in that, include: The module is used to unify space-based and ground-based data into the same geographic coordinate system, extract the spatial location, timestamp, and sensor physical quantities of data points, and construct a multi-dimensional feature vector set. The calculation module is used to construct a data index tree based on the multi-dimensional feature vector set. During the node splitting process, it calculates and stores the covariance matrix and inverse matrix of the data point set within the node. When the ratio of the largest eigenvalue to the second largest eigenvalue of the covariance matrix is ​​greater than the structure discrimination threshold, the structure discrimination threshold increases monotonically with the node depth and is constrained by a preset upper limit threshold. The oriented capsule with the first principal component vector as the central axis is used as the bounding body of the node; otherwise, the minimum bounding sphere is used as the bounding body. The generation module receives the current air data point as the query point, calculates the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate node in the tree, and converts it into a global distance. It initializes the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, places the nodes in the region at the front of the queue to be queried, and removes the candidate node if the distance between the query point and the boundary of the bounding body of the candidate node is greater than the optimal nearest neighbor distance. It records the node in the queue whose boundary of the bounding body is closest to the query point as the second-best candidate node. If the approximate nearest neighbor distance is greater than the boundary distance of the second-best candidate node, it performs a local search to update the nearest neighbor of the second-best candidate node, and generates data records based on the association between the nearest neighbor and the query point.

2. The system according to claim 1, characterized in that, The spatial location, timestamp, and sensor physical quantities of the extracted data points are used to construct a multi-dimensional feature vector set, including: Extract the three-dimensional spatial coordinates of space-based and ground-based data points in a unified coordinate system as their three-dimensional spatial positions; Convert the data collection time into a numerical timestamp scalar; Temperature, relative humidity, and carbon dioxide concentration values ​​of fixed dimensions are extracted as physical quantities of the sensor. The three-dimensional spatial position, timestamp scalar, and sensor physical quantity are respectively processed by dimensionless normalization and then concatenated in order to construct a multi-dimensional floating-point array as a feature vector.

3. The system according to claim 2, characterized in that, The covariance matrix and inverse matrix of the data point set within the computing node are stored, including: Calculate the multidimensional mean vector of the feature vectors of all data points within the current node; Subtract the multidimensional mean vector from the feature vector of each data point to obtain the offset vector of each data point; Calculate the sum of the products of all offset vectors and the transpose matrix, and divide by the total number of data points within the node to generate the standard covariance matrix; Add positive constants to each element of the main diagonal of the standard covariance matrix to generate a non-singular positive definite matrix; The positive definite matrix is ​​inverted using the Gauss-Jordan elimination algorithm to obtain a two-dimensional floating-point inverse matrix, which is then cached and recorded as a unique attribute variable for the current node.

4. The system according to claim 1, characterized in that, The structure discrimination threshold increases monotonically with increasing node depth and is constrained by a preset upper limit threshold, including: Set the node depth of the root node of the index tree to 0, and set the initial structure discrimination threshold constant and a preset upper limit threshold greater than the initial structure discrimination threshold constant; Set a base that is greater than 1 and increases progressively at each level; During node splitting, the node depth integer D of the current target node is obtained, the initial structure discrimination threshold is multiplied by the D power of the increasing base, and the minimum value is taken with the preset upper limit threshold to calculate and generate a single-precision floating-point value, which is used as the structure discrimination threshold of the current node.

5. The system according to claim 1 or 4, characterized in that, The method of using a directional capsule with the first principal component vector as the central axis as the bounding volume of the node includes: Singular value decomposition is performed on the covariance matrix, and the unit vector corresponding to the largest eigenvalue is extracted as the first principal component vector; Calculate the scalar value of the projection of the feature vector of each data point within the node onto the multidimensional spatial axis where the first principal component vector is located; Obtain the projection points corresponding to the maximum and minimum projection values, and take the closed spatial line segment between the two extreme projection points as the central axis segment of the directional capsule. The orthogonal distance from the eigenvector of each data point within a node to the central axis segment is calculated by combining the inverse matrix of the covariance matrix. The largest orthogonal distance is selected as the multidimensional unified surface boundary radius of the directional capsule, and the directional capsule is generated.

6. The system according to claim 1, characterized in that, The step of calculating the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate nodes of the tree includes: Load the inverse matrix M stored in the attributes of the current candidate node or the second-best candidate node; Obtain the feature vector Q of the query point and the feature vector P of the test point, calculate the feature difference at the corresponding index position, and generate the difference vector. The transpose of the difference vector, the inverse matrix M, and the difference vector are multiplied sequentially to obtain the scalar sum of the products; The square root of the sum of the scalars is taken to obtain the local adaptive Mahalanobis distance between the query point and the test point.

7. The system according to claim 1, characterized in that, The step of initializing the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, and placing the nodes within the region at the front of the query queue, includes: Get the nearest neighbor leaf node number confirmed after the query point completed the search at the previous time step; In the tree structure, trace upwards two levels to find the grandparent node of the nearest neighbor leaf node, and divide the space under the bounding volume of the grandparent node into a priority search region. Read the sequence of all child nodes and all leaf nodes under the grandfather node, and insert all nodes in the sequence into the front of the system's query queue according to the storage order, as the first batch of test nodes.

8. The system according to claim 1, characterized in that, The step of performing a local search to update the nearest neighbor for the suboptimal candidate node includes: Extract the recorded suboptimal candidate nodes, or the set of all underlying raw data points contained in the current compensation candidate node extracted from the unvisited candidate node set; Traverse the underlying set of original data points and calculate the local adaptive Mahalanobis distance between the query point and each original data point; Find the minimum value among the calculated local adaptive Mahalanobis distances. If the minimum value is less than the current best nearest neighbor distance, then take the original data point corresponding to the minimum value as the updated current best nearest neighbor point, and update the current best nearest neighbor distance simultaneously.

9. A method for intelligent data fusion processing across multiple terminals in air, space, and ground, characterized in that, include: Unify space-based and ground-based data into the same geographic coordinate system, extract the spatial location, timestamp, and sensor physical quantities of data points, and construct a multi-dimensional feature vector set; A data index tree is constructed based on the multi-dimensional feature vector set. During the node splitting process, the covariance matrix and inverse matrix of the data point set within the node are calculated and stored. When the ratio of the largest eigenvalue to the second largest eigenvalue of the covariance matrix is ​​greater than the structure discrimination threshold, the structure discrimination threshold increases monotonically with the node depth and is constrained by a preset upper limit threshold. An oriented capsule with the first principal component vector as the central axis is used as the bounding body of the node; otherwise, the minimum bounding sphere is used as the bounding body. The system receives the current air data point as the query point, calculates the local adaptive Mahalanobis distance between the query point and the candidate node using the inverse matrix of the covariance matrix stored in the candidate node in the tree, and converts it into a global distance. It initializes the priority search region using the leaf node where the nearest neighbor of the query point is located at the previous time step, places the nodes in the region at the front of the queue to be queried, and removes the candidate node if the distance between the query point and the boundary of the bounding body of the candidate node is greater than the optimal nearest neighbor distance. It records the node in the queue whose boundary of the bounding body is closest to the query point as the second-best candidate node. If the approximate nearest neighbor distance is greater than the boundary distance of the second-best candidate node, it performs a local search to update the nearest neighbor of the second-best candidate node, and generates data records based on the association between the nearest neighbor and the query point.

10. The method according to claim 9, characterized in that, The spatial location, timestamp, and sensor physical quantities of the extracted data points are used to construct a multi-dimensional feature vector set, including: Extract the three-dimensional spatial coordinates of space-based and ground-based data points in a unified coordinate system as their three-dimensional spatial positions; Convert the data collection time into a numerical timestamp scalar; Temperature, relative humidity, and carbon dioxide concentration values ​​of fixed dimensions are extracted as physical quantities of the sensor. The three-dimensional spatial position, timestamp scalar, and sensor physical quantity are respectively processed by dimensionless normalization and then concatenated in order to construct a multi-dimensional floating-point array as a feature vector.

Images