A visual situation display method, device, equipment and storage medium

Abstract

The application discloses a visual situation display method and device, equipment and a storage medium. Multi-source heterogeneous sensor data is acquired and analyzed, standardized observation data is obtained through standardization processing, the standardized observation data is unified in time and space reference, time-space aligned data and uncertainty measurement data are generated, multi-level data correlation and sequential fusion are carried out based on the uncertainty measurement data, multi-source fusion target tracks are obtained, dynamic target group identification and hierarchical aggregation are carried out on the multi-source fusion target tracks, aggregated situation data and target display priority data are obtained, the aggregated situation data and the fusion tracks are dynamically rendered in combination with viewport information and priority data, a multi-dimensional situation visual interface is constructed, real-time optimization parameters are obtained based on interface quality indexes and operator interactive behavior data, and data processing and rendering strategies are dynamically adjusted. The application realizes closed-loop collaborative optimization of data processing and rendering presentation.

Description

Technical Field

[0001] This application relates to the field of data visualization technology, and in particular to a visualization situation display method, apparatus, device and storage medium. Background Technology

[0002] As command and control systems, industrial monitoring, and complex environment monitoring systems continue to expand in scale, more and more large or ultra-large multi-source heterogeneous sensors and data sources are being integrated into these systems. However, the data environments faced by these systems are often extremely complex. When processing massive amounts of dynamic situational data, operators are prone to difficulty in quickly and accurately grasping the overall situation due to asynchronous data, spatial deviations, information overload, and a lack of dynamic representation. This can even lead to the omission of key target information, resulting in significant time being spent subsequently searching for and verifying targets, thereby reducing decision-making and response efficiency. To address this issue, it is necessary to perform precise spatiotemporal alignment, intelligent aggregation, and intuitive visualization of multi-source heterogeneous data.

[0003] Current methods typically employ traditional timestamp matching, static coordinate transformation, nearest neighbor data association, and fixed-layer 2D maps for situational awareness display. These methods usually rely on simple interpolation or pre-calibrated parameters and employ fixed timed refresh and overlay rendering strategies. However, under complex conditions such as large differences in sensor sampling frequencies, high target density, or dynamically changing environments, these methods exhibit poor adaptability and coverage, easily leading to problems such as lost time alignment, large spatial accumulation errors, incorrect data association, and chaotic interface information. This results in reduced accuracy and real-time performance in situational awareness representation and target identification.

[0004] Therefore, how to achieve high-precision spatiotemporal alignment and intelligent aggregation of multi-source heterogeneous data, and improve the accuracy, clarity and adaptive interaction efficiency of situation display, is an urgent problem to be solved. Summary of the Invention

[0005] In view of this, the visualization situation display method, apparatus, device, and storage medium provided in this application can achieve high-precision spatiotemporal alignment and intelligent aggregation of multi-source heterogeneous data, improving the accuracy, clarity, and adaptive interaction efficiency of situation display. The visualization situation display method, apparatus, device, and storage medium provided in this application are implemented as follows: This application provides a method for visualizing situational awareness, comprising: Acquire multi-source heterogeneous sensor data, and perform analysis and standardization processing on the multi-source heterogeneous sensor data to obtain standardized observation data; The standardized observation data is processed to unify the time reference and the spatial reference, resulting in spatiotemporally aligned data and corresponding uncertainty measurement data. Based on the uncertainty measurement data, the spatiotemporal aligned data is subjected to multi-level data association and sequential fusion processing to obtain multi-source fused target tracks; Dynamic target group identification and hierarchical aggregation processing are performed on the multi-source fusion target trajectory to obtain aggregated situational data and target display priority data; Based on the target display priority data and viewport information, the aggregated situation data and the multi-source fusion target trajectory are dynamically rendered to obtain a multi-dimensional situation visualization interface. The visualization interface quality index and operator interaction behavior data of the multi-dimensional situation visualization interface are obtained. Based on the visualization interface quality index and operator interaction behavior data, optimization processing is performed to obtain real-time optimization parameters. Based on the real-time optimization parameters, the data processing and rendering strategies are dynamically adjusted.

[0006] In some embodiments, acquiring multi-source heterogeneous sensor data and parsing and standardizing the multi-source heterogeneous sensor data to obtain standardized observation data includes: The multi-source heterogeneous sensor data is identified and analyzed to obtain analyzed sensor data. The multi-source heterogeneous sensor data includes at least two of the following: radar, photoelectric, electronic reconnaissance, satellite navigation, communication link, meteorology, and artificial intelligence. The parsed sensor data is subjected to integrity detection processing to obtain valid data; based on the signal-to-noise ratio and detection probability calculation index, the valid data is subjected to data quality assessment processing to obtain data credibility assessment results; Identify and remove outlier data points from the valid data to obtain corrected observation data; Based on the data credibility assessment results, the attribute fields and data types of the corrected observation data are normalized and transformed to obtain standardized observation data.

[0007] In some embodiments, the process of unifying the time and space references of the standardized observation data to obtain spatiotemporally aligned data and corresponding uncertainty metric data includes: The standardized observation data are subjected to time transformation processing to obtain a unified time series; Based on the difference in sensor sampling frequency, the unified time series is resampled to obtain time-aligned data, and the interpolation error covariance matrix is ​​calculated synchronously. Transform the local coordinates of each sensor to the geocentric-ground-fixed coordinate system to obtain the initial geocentric-ground-fixed coordinates; A dynamic spatial correction model is constructed based on extended Kalman filtering, which models sensor bias as a state variable and uses multi-sensor cross-validation information to update correction parameters online. Based on the updated correction parameters, the initial geocentric and geofixed coordinates are corrected and transformed to obtain spatially corrected geographic coordinate data. The time-aligned data, the interpolation error covariance matrix, and the spatially corrected geographic coordinate data are merged and integrated to obtain spatiotemporal aligned data and corresponding uncertainty measurement data.

[0008] In some embodiments, the step of performing multi-level data association and sequential fusion processing on the spatiotemporal aligned data based on the uncertainty metric data to obtain a multi-source fused target trajectory includes: Based on spatial distance and velocity characteristics, the spatiotemporal alignment data is subjected to coarse correlation filtering to obtain a set of candidate correlation targets. Extract the multidimensional feature vectors of the candidate associated target set, and perform fine association calculation based on the Mahalanobis distance using the multidimensional feature vectors to obtain the association matrix; The correlation matrix is ​​solved using the Hungarian algorithm or the auction algorithm to obtain the globally optimal data correlation result; Based on the global optimal data association results and the uncertainty measurement data, the spatiotemporal aligned data is sequentially fused in chronological order to obtain the multi-source fused target track.

[0009] In some embodiments, the step of performing dynamic target group identification and hierarchical aggregation processing on the multi-source fused target trajectory to obtain aggregated situational data and target display priority data includes: A density-based clustering algorithm is used to identify target groups in the multi-source fusion target tracks. The clustering parameters are adaptively adjusted according to the target distribution density, and the stability of the groups is evaluated by combining the structural consistency at historical moments to obtain the target group division results. Based on the target group division results, the multi-source fusion target tracks are subjected to multi-scale hierarchical aggregation processing in combination with the viewport scaling ratio and spatial distribution characteristics to generate group or target representation forms corresponding to the display level, thereby obtaining aggregated situational data. Based on the target's attribute characteristics, timeliness characteristics, and operator interaction characteristics, a multi-dimensional scoring model is used to calculate and sort the priority of each target and target group, thus obtaining target display priority data.

[0010] In some embodiments, the step of dynamically rendering the aggregated situational data and the multi-source fused target trajectory based on the target display priority data and viewport information to obtain a multi-dimensional situational visualization interface includes: The spatiotemporal characteristics of the multi-source fusion target trajectory are graphically mapped to obtain trajectory situation representation data; The aggregated situational data and the situational elements in the multi-source fusion target trajectory are subjected to multi-dimensional perception attribute fusion visual representation processing to obtain information representation data; The target groups in the aggregated situational data are processed to macroscopically express their group distribution characteristics and overall evolution trends, resulting in group situational expression data. Based on the viewport information and the target display priority data, multi-scale adaptive rendering control processing is performed on the trajectory situation expression data, the information representation data, and the group situation expression data to obtain a multi-dimensional situation visualization interface.

[0011] In some embodiments, the step of acquiring the visualization interface quality index and operator interaction behavior data of the multi-dimensional situation visualization interface, performing optimization processing based on the visualization interface quality index and the operator interaction behavior data to obtain real-time optimization parameters, and dynamically adjusting the data processing and rendering strategy based on the real-time optimization parameters includes: The visual presentation effect of the multi-dimensional situation visualization interface is evaluated to obtain the visualization interface quality index. The operator's interface interaction actions are tracked and their intent is inferred to obtain operator interaction behavior data; Based on the quality indicators of the visual interface and the operator interaction behavior data, a comprehensive optimization calculation is performed to obtain real-time optimization parameters. Based on the aforementioned real-time optimization parameters, the clustering algorithm parameters and fusion processing accuracy in the data processing stage, as well as the resource allocation strategy and interface refresh rate in the rendering stage, are dynamically adjusted.

[0012] This application provides a visualization situation display device, comprising: The acquisition module is used to acquire multi-source heterogeneous sensor data, and to parse and standardize the multi-source heterogeneous sensor data to obtain standardized observation data. The processing module is used to perform time benchmark unification and spatial benchmark unification processing on the standardized observation data to obtain spatiotemporally aligned data and corresponding uncertainty measurement data; The processing module is also used to perform multi-level data association and sequential fusion processing on the spatiotemporal alignment data based on the uncertainty measurement data to obtain multi-source fusion target tracks; The processing module is also used to perform dynamic target group identification and hierarchical aggregation processing on the multi-source fusion target trajectory to obtain aggregated situational data and target display priority data; The processing module is also used to dynamically render the aggregated situation data and the multi-source fusion target trajectory based on the target display priority data and viewport information to obtain a multi-dimensional situation visualization interface. The interaction module is used to acquire the visualization interface quality indicators and operator interaction behavior data of the multi-dimensional situation visualization interface, perform optimization processing based on the visualization interface quality indicators and operator interaction behavior data to obtain real-time optimization parameters, and dynamically adjust the data processing and rendering strategies based on the real-time optimization parameters.

[0013] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0014] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method provided in this application embodiment.

[0015] This application provides a visualization situation display method, apparatus, device, and storage medium. It acquires and parses multi-source heterogeneous sensor data, and obtains standardized observation data through standardization processing. The standardized observation data undergoes spatiotemporal benchmark unification to generate spatiotemporal aligned data and uncertainty measurement data. Based on the uncertainty measurement data, multi-level data association and sequential fusion are performed to obtain multi-source fused target tracks. Dynamic target group identification and hierarchical aggregation are performed on the multi-source fused target tracks to obtain aggregated situation data and target display priority data. The aggregated situation data and fused tracks are dynamically rendered using viewport information and priority data to construct a multi-dimensional situation visualization interface. Real-time optimization parameters are obtained based on interface quality indicators and operator interaction behavior data to dynamically adjust data processing and rendering strategies. This enables high-precision spatiotemporal alignment and intelligent aggregation of multi-source heterogeneous data, improving the accuracy, clarity, and adaptive interaction efficiency of situation display, and solving the technical problems mentioned in the background art. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 A schematic diagram illustrating the implementation process of a visualization situation display method provided in this application embodiment; Figure 2 This application provides a schematic diagram of an implementation process for acquiring standardized observation data. Figure 3This is a schematic diagram of the structure of a visualization situation display device provided in an embodiment of this application. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0019] The following description of some technologies involved in the embodiments of this application is provided to aid understanding and should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and brevity, some descriptions of well-known functions and structures are omitted in the following description.

[0020] Figure 1 This is a schematic diagram illustrating the implementation flow of a visualization situation display method provided in an embodiment of this application, including steps 101 to 106. Wherein, Figure 1 This is merely one execution order shown in the embodiments of this application and does not represent the only execution order for a visualization situation display method. Where the final result can be achieved, Figure 1 The steps shown can be performed in parallel or in reverse order.

[0021] Step 101: Acquire multi-source heterogeneous sensor data, analyze and standardize the multi-source heterogeneous sensor data to obtain standardized observation data.

[0022] In this embodiment, the system is equipped with a plug-in data adaptation layer as a unified data acquisition interface. Dedicated parsing plug-ins are independently developed for different data source types, supporting dynamic loading of plug-ins and automatic identification of data sources. The system can access at least two different types of multi-source heterogeneous sensor data, including radar (primary / secondary / phased array), photoelectric (visible light / infrared / laser ranging), electronic reconnaissance, satellite navigation (GPS / BeiDou), communication links, meteorology, and artificial intelligence. The system automatically identifies the data stream protocol type, dynamically loads the corresponding parsing plug-in, and extracts the timestamp, three-dimensional position, motion attributes, measurement noise, and original sensor status information from the message.

[0023] The received raw data stream is then subjected to a four-layer normalization preprocessing: 1. Integrity verification: Detects packet loss, out-of-order data, and missing fields; filters illegally transmitted data; and outputs complete and valid data.

[0024] 2. Data quality assessment: Calculate a quantitative score for credibility based on signal-to-noise ratio and sensor detection probability to generate data credibility assessment results.

[0025] 3. Outlier removal: 3-Sigma and sliding window mid-value filtering are used to identify and remove transient interference and environmental clutter-induced anomalous points, and the corrected observation data is output.

[0026] 4. Standardized format conversion: Based on a unified data dictionary, the naming of heterogeneous fields, physical dimensions, and data precision are standardized, and the aforementioned credibility assessment results are written into the data structure as uncertainty labels to output standardized observation data.

[0027] Step 102: Perform time benchmark unification and spatial benchmark unification processing on the standardized observation data to obtain spatiotemporally aligned data and corresponding uncertainty measurement data.

[0028] In this embodiment, microsecond-level time synchronization and dynamic adaptive spatial error compensation are fully realized, and interpolation and coordinate transformation uncertainty measurement data are generated synchronously. This is divided into two major sub-processes: time base unification and spatial base unification. 1. Unified time reference (PTP (Precision Time Protocol) + GPS / BeiDou backup + UTC unified reference + cubic spline interpolation resampling).

[0029] The IEEE 1588PTP precision time protocol is preferred for synchronizing the sensors with the ground station at the microsecond level. For sensors that cannot deploy PTP, GPS / BeiDou PPS pulse timing is used as an alternative, with a synchronization accuracy of up to 100 nanoseconds. All local clocks of the sensors are uniformly converted to the UTC universal coordinated time reference to generate a unified time series.

[0030] Cubic spline interpolation resampling: To address the difference in sampling periods between high- and low-frequency sensors (radar 1Hz, photoelectric 30Hz), a globally unified time grid is constructed, and cubic spline interpolation is used to complete time-series resampling: low-frequency data is interpolated between sampling points to supplement intermediate time state estimation, and high-frequency data is down-frequency matched to the unified grid; the interpolation error covariance matrix is ​​derived simultaneously during the interpolation process to quantify the uncertainty introduced by time-series alignment.

[0031] 2. Spatial reference unification (EKF (Extended Kalman Filter) dynamic correction, covering all sources of dynamic error).

[0032] Local coordinates to ECEF (Earth-Centered Earth-Fixed) coordinates: Based on the WGS-84 ellipsoid model, local measurements such as radar station polar coordinates and electro-optical azimuth and elevation coordinates are uniformly converted into three-dimensional Earth-Centered Earth-Fixed coordinates to obtain the initial ECEF coordinates.

[0033] Extended Kalman filter dynamic spatial correction model: All dynamic error parameters, including sensor installation deviation, carrier attitude change, atmospheric refraction, multipath propagation effect, and zero drift, are modeled as filter state variables; the correction parameters are updated online recursively using the cross-validation residuals from multi-sensor overlapping observations as the observation update quantity, without the need for task interruption and recalibration.

[0034] Secondary coordinate transformation: The updated correction parameters are used to compensate for the initial ECEF coordinate system error, and then converted to WGS-84 latitude, longitude and altitude geographic coordinates.

[0035] 3. Uncertainty integration output.

[0036] By merging and overlaying time-aligned data, cubic spline interpolation error covariance matrix, corrected geographic coordinates, and spatial transformation error matrix, spatiotemporal aligned data and accompanying uncertainty measurement data with complete error representation are generated.

[0037] Step 103: Based on the uncertainty measurement data, perform multi-level data association and sequential fusion processing on the spatiotemporal aligned data to obtain the multi-source fused target track.

[0038] In this embodiment, the measurement noise distribution is quantified by relying on the uncertainty metric covariance matrix, and a three-layer progressive data association is performed to solve the problems of mismatch in high-density target association and computational explosion. 1. Coarse correlation screening: Based on the Euclidean distance threshold and velocity vector difference threshold in three-dimensional space, irrelevant observation pairs are quickly filtered out. The threshold is dynamically adjusted according to three times the standard deviation of the sensor's nominal accuracy, which quickly narrows down the candidate correlation set and reduces the computational dimensionality.

[0039] 2. Fine-grained association using Mahalanobis distance (expanded to include complete multi-dimensional feature vectors): Extracts multi-dimensional feature vectors of candidate targets, including ① motion features: 3D position, velocity, acceleration, and heading; ② physical features: radar cross-section, infrared radiation intensity, and signal spectrum features; ③ temporal features: first detection time and continuous tracking duration; Mahalanobis distance is calculated using the uncertainty covariance matrix to quantify feature similarity and generate an association cost matrix; in high-dimensional feature scenarios, dimensionality reduction can be used to compress computational costs.

[0040] 3. Global optimal association solution: The association matching is transformed into a bipartite graph assignment problem, and the Hungarian algorithm is used to solve the global one-to-one optimal matching. For ultra-large scenarios with more than 1,000 targets, the auction algorithm can be replaced to improve the parallel computing efficiency and avoid the local optimality defect of the traditional nearest neighbor association.

[0041] 4. Sequential filtering fusion: Based on the global optimal correlation results, the uncertainty metric data is used as the basis for the allocation of filtering weights. Sequential Kalman filtering is performed in order of timestamps, updating the target position, velocity, and covariance state frame by frame, and outputting a smooth, continuous, and low-jitter multi-source fused target track.

[0042] Step 104: Perform dynamic target group identification and hierarchical aggregation processing on the multi-source fusion target tracks to obtain aggregated situational data and target display priority data.

[0043] In this embodiment of the application, an adaptive aggregation system is constructed to address the problem of massive and dense target interface stacking and information overload: An improved density-based DBSCAN (Density-Based Spatial Clustering of Applications with Noise) clustering algorithm is used to identify groups of co-moving targets. The neighborhood radius and minimum number of points clustering parameters are adaptively adjusted based on the local target distribution density. A temporal group stability evaluation mechanism is introduced to compare the consistency of group structure in consecutive frames and suppress frequent group splitting and reconstruction caused by single-frame data jitter. Stable group partitioning results are output.

[0044] The system combines map viewport zoom level and group spatial distribution range to achieve multi-scale hierarchical representation: macro small-scale view aggregates groups into group outlines with statistical labels (target quantity, average speed, overall heading); medium-scale view is split into sub-groups for mixed display; micro large-scale view fully renders all independent targets; the hierarchical switching configuration has smooth animation transition without abrupt jumps; and hierarchical aggregated situational data is generated.

[0045] A comprehensive scoring model is constructed to calculate the display priority of targets / groups. The input dimensions include: the inherent threat attributes of the target, the timeliness and freshness of the track information, and the operator's interaction focus (selection, hovering, retrieval). High-priority targets are highlighted, bolded, and have their complete tracks rendered. Low-priority targets have their macroscopic views automatically aggregated and simplified. The output is target display priority data.

[0046] Step 105: Based on the target display priority data and viewport information, perform dynamic rendering processing on the aggregated situation data and multi-source fusion target tracks to obtain a multi-dimensional situation visualization interface.

[0047] In this embodiment, layered adaptive rendering is performed by combining viewport resolution, scaling level, display boundary, and target priority data: Graphical mapping of tracks: The spatiotemporal features of the track are transformed into visual primitives. Gradient color track tail lines express the motion history, vector arrows represent the predicted velocity direction, and dashed lines extrapolate the future predicted trajectory.

[0048] Multi-channel visual fusion representation: Sensor confidence, data source, alarm status are mapped to visual channels such as color transparency, symbol line type, and dynamic flashing, and multi-dimensional information is superimposed without text redundancy.

[0049] Macro-level group situation rendering: α-Shape convex hull is used to extract group boundaries, heat map is used to represent target distribution density, and streamline vector is used to express the overall aggregation / diffusion evolution trend of the group.

[0050] Multi-scale rendering control: Performs frustum culling, rendering only primitives within the viewport; dynamically allocates GPU / CPU computing power according to priority, fully rendering labels and trajectory details for high-priority targets, and simplifying low-priority targets into point primitives; primitives automatically avoid occlusion, ensuring high interface smoothness and generating a multi-dimensional situational visualization interface.

[0051] Step 106: Obtain the visualization interface quality indicators and operator interaction behavior data of the multi-dimensional situation visualization interface, perform optimization processing based on the visualization interface quality indicators and operator interaction behavior data to obtain real-time optimization parameters, and dynamically adjust the data processing and rendering strategies based on the real-time optimization parameters.

[0052] In this embodiment, a closed-loop adaptive feedback mechanism is constructed that spans the entire data processing, visualization rendering, and human-computer interaction chain. The core mechanism employs Bayesian optimization to achieve automatic parameter tuning. Real-time visual quality assessment: Continuously monitor interface rendering frame rate, CPU / GPU usage, primitive overlap and occlusion, primitive congestion per unit area, and interface display latency, and quantify and generate visual interface quality indicators.

[0053] Operator interaction behavior tracking and intent inference: Real-time collection of panning, zooming, box selection, target point selection, and layer switching interaction actions; inference of operator attention areas and key target types through sliding window time sequence analysis; output of interaction behavior data.

[0054] Bayesian optimization comprehensive tuning calculation: Taking the comprehensive score of fusion accuracy, interface rendering latency and situation visualization clarity as the objective function, the Bayesian optimization algorithm is used to solve the optimal real-time tuning parameters asynchronously in the background without occupying the real-time processing computing power of the foreground.

[0055] End-to-end dynamic strategy adjustment: Optimization parameters are back-synchronized to the front-end data processing and back-end rendering engine to achieve multi-level adaptive adjustment. Data processing layer: dynamically modifies DBSCAN clustering neighborhood radius, sequential filtering update step size, and fusion accuracy threshold.

[0056] Rendering layer: dynamically allocates computing resources, adjusts the interface refresh rate, and modifies the threshold for detail levels.

[0057] Improve the fusion accuracy, rendering details, and refresh rate for areas of focus for operators; moderately downsample and aggregate non-focus areas to save computing resources and form a closed-loop collaborative optimization.

[0058] This application's embodiments establish for the first time a closed-loop mechanism encompassing the entire chain of "data parsing / spatiotemporal alignment -> correlation fusion -> group aggregation -> priority rendering -> interactive feedback." By using interface rendering quality and operator interaction intent to drive data processing and rendering strategies, it solves the problem of static configuration and data / rendering disconnect in traditional systems. It achieves high-precision alignment and efficient rendering of massive dynamic data, significantly improving the accuracy, clarity, and smoothness of dynamic situational awareness display in complex, large-scale scenarios, while reducing the information cognitive load on operators.

[0059] In the above Figure 1 Based on the above, this application embodiment also provides a schematic diagram of the implementation process for obtaining standardized observation data. For example... Figure 2 As shown, steps 201 to 204 are included: Step 201: Identify and analyze the multi-source heterogeneous sensor data to obtain the analyzed sensor data.

[0060] In this embodiment, the data acquisition module connects to multiple data sources with different physical attributes and transmission protocols. These multi-source heterogeneous sensor data include at least two of the following: radar, photoelectric, electronic reconnaissance, satellite navigation, communication links, meteorology, and artificial intelligence. Because the data message formats of each sensor differ (such as binary streams, XML, JSON, or custom protocols), the system uses a preset protocol parsing rule base to identify the protocol and unpack the message structure of the incoming raw data stream, extracting the target position, velocity, angle, timestamp, measurement error parameters, and sensor status information to obtain the parsed sensor data.

[0061] Step 202: Perform integrity detection processing on the parsed sensor data to obtain valid data; based on the signal-to-noise ratio and detection probability calculation index, perform data quality assessment processing on the valid data to obtain the data credibility assessment result.

[0062] In this embodiment, the parsed sensor data is first subjected to data integrity verification. Key attribute fields (such as timestamps and spatial coordinates) are checked for missing data, garbled characters, or packet loss during transmission. Illegal data that does not meet integrity requirements is removed, and valid data is filtered out. Based on this, the system establishes a data quality assessment model to extract signal quality indicators from the valid data, including signal-to-noise ratio and sensor detection probability. Combining this with a preset quality assessment function, the system comprehensively calculates the data quality score and uncertainty limit of each sensor under the current observation environment, obtaining a data reliability assessment result that quantitatively characterizes the reliability of this batch of data.

[0063] Step 203: Identify and remove outlier data points from the valid data to obtain corrected observation data.

[0064] In this embodiment, based on the continuity of effective data in time and the physical kinematics, outlier filtering algorithms (such as 3-Sigma principle, statistical outlier detection, or sliding window midpoint filtering) are used to dynamically screen the effective data; outliers and jump points that exceed reasonable physical limits due to sensor transient interference, environmental clutter, or transmission errors are identified and removed, eliminating the impact of outlier noise on subsequent fusion calculations, and obtaining corrected observation data.

[0065] Step 204: Based on the data credibility assessment results, the attribute fields and data types of the corrected observation data are normalized and transformed to obtain standardized observation data.

[0066] In this embodiment, the system maps and normalizes heterogeneous attribute fields (such as different naming rules, unit dimensions, and data precision) from different sensors in the modified observation data according to a preset standard data dictionary, and converts them into a standard data structure. At the same time, the calculated data credibility assessment results are used as additional credibility weights or uncertainty labels and are synchronously written into the normalized data structure. Finally, the system outputs standardized observation data with a standard format, unified dimensions, and quality calibration for subsequent spatiotemporal alignment and data fusion steps.

[0067] This application's embodiments ensure the reliability of input data from the source by performing integrity detection, outlier filtering (anomaly removal), and quality assessment of signal-to-noise ratio and detection probability on multi-source heterogeneous sensor data. Normalization not only resolves the protocol and structural differences between heterogeneous data but also adds a quantitative data credibility label to subsequent fusion calculations, preventing inferior data from contaminating the subsequent fusion algorithm.

[0068] In some embodiments, standardized observation data are subjected to time and spatial benchmark unification processing to obtain spatiotemporally aligned data and corresponding uncertainty measurement data, including: performing time transformation processing on standardized observation data to obtain a unified time series.

[0069] Specifically, since the internal clocks of each sensor may have initial deviations, drifts, or use different time scales (such as local high-precision timers, private timestamps of each sensor, etc.), the system first reads the original timestamps in the standardized observation data; then, according to the clock synchronization protocol and time offset of each sensor, the asynchronous timestamps from different sensors are uniformly mapped and converted to a preset universal time reference (such as Coordinated Universal Time UTC) to obtain a time series aligned under a unified time reference.

[0070] Furthermore, based on the difference in sensor sampling frequency, the unified time series is resampled to obtain time-aligned data, and the interpolation error covariance matrix is ​​calculated simultaneously.

[0071] Specifically, considering the significant differences in sampling periods and data update frequencies among different types of sensors (such as high-frequency radar and low-frequency optoelectronic devices), the system sets a unified synchronization time node (such as a fixed frame rate period). Based on the differences in sampling frequencies, high-order interpolation algorithms (such as Lagrange interpolation, cubic spline interpolation, or extrapolation prediction algorithms) are used to resample and dynamically fit the observed states (such as position and velocity) in the unified time series, aligning the observed states of each sensor to the same synchronous sampling time and generating time-aligned data. At the same time, an interpolation uncertainty propagation model is established to derive and calculate the interpolation error covariance matrix introduced by the time-series interpolation or extrapolation process, which is used to quantitatively characterize the state uncertainty in the time alignment process.

[0072] Furthermore, the local coordinates of each sensor are transformed to the geocentric coordinate system to obtain the initial geocentric coordinates.

[0073] Specifically, the spatial position input by each sensor is usually based on its own local coordinate system (such as polar coordinate system, site-relative rectangular coordinate system, etc.). The system uses prior installation parameters such as the geographical latitude and longitude, elevation angle, and installation azimuth angle of each sensor site to establish a coordinate transformation matrix, and uniformly transforms the target measurement data in the local coordinate system of each sensor to the globally universal geocentric coordinate system, and solves for the initial geocentric coordinates in three-dimensional space.

[0074] Furthermore, a dynamic spatial correction model is constructed based on extended Kalman filtering, which models sensor bias as a state variable and uses multi-sensor cross-validation information to update correction parameters online.

[0075] Specifically, considering that sensors may have fixed installation position deviations, attitude angle system errors, or zero-point drift in actual deployment, the system constructs a dynamic spatial correction model based on extended Kalman filtering; it establishes the sensor's position deviation and angle deviation and other system errors as state vectors; when multiple sensors conduct overlapping observations of the same target or the same area, it extracts the cross-validation residual information of multi-source observations as the observation update quantity of Kalman filtering, and recursively estimates and dynamically updates the sensor's system correction parameters online.

[0076] Furthermore, the initial geocentric coordinates are corrected and transformed based on the updated correction parameters to obtain spatially corrected geographic coordinate data.

[0077] Specifically, the online estimated and updated correction parameters are fed back to the spatial transformation calculation in real time to perform systematic error compensation and dynamic correction on the initial geocentric and geofixed coordinates obtained by the aforementioned calculation, eliminating the inherent installation error and measurement deviation of the sensor; further, the corrected three-dimensional geocentric and geofixed coordinates are transformed to a standard geographic coordinate system (such as longitude, latitude and altitude under the WGS-84 coordinate system) to obtain highly spatially corrected geographic coordinate data.

[0078] Furthermore, the time-aligned data, interpolation error covariance matrix, and spatially corrected geographic coordinate data are merged and integrated to obtain spatiotemporally aligned data and corresponding uncertainty measurement data.

[0079] Specifically, the time-aligned data after time-dimensional alignment is fused with the geographic coordinate data after spatial-dimensional correction to form a complete observation state vector that is fully aligned in both time and space. At the same time, the covariance matrix of the interpolation error is superimposed with the sensor's inherent measurement noise matrix and the spatial coordinate transformation error matrix to construct a comprehensive error covariance matrix that fully reflects the quality of time-space alignment and measurement accuracy, which serves as uncertainty measurement data. Finally, the two are integrated and bound together for output, resulting in time-space aligned data and uncertainty measurement data containing the time-space alignment status and its corresponding accuracy label, which can be used for subsequent multi-level data association and fusion calculations.

[0080] This application's embodiments address the issues of inconsistent sampling periods and spatial reference system differences among multi-source sensors by combining unified time-series resampling with geocentric-geostationary coordinate system transformation. An extended Kalman filter is introduced to estimate and correct sensor system biases online, significantly reducing geometric accumulation errors. Simultaneously, the interpolation error covariance and coordinate transformation error are derived, providing accurate uncertainty measurement support for data correlation.

[0081] In some embodiments, based on uncertainty measurement data, multi-level data association and sequential fusion processing is performed on spatiotemporal aligned data to obtain multi-source fused target tracks, including: performing coarse association screening processing on spatiotemporal aligned data based on spatial distance and velocity characteristics to obtain a candidate associated target set.

[0082] Specifically, in scenarios with massive amounts of observation data, directly performing full-combination fine-grained matching would lead to extremely high computational complexity. First, a coarse correlation screening mechanism is established (such as a grid method based on spatial partitioning or motion beamgating constraints). Then, using the difference between the three-dimensional spatial location distance (such as Euclidean distance) and velocity vector in the spatiotemporally aligned data, fast distance and velocity gates are used to quickly filter out heterogeneous observation pairs that are too far apart or whose motion trends are completely mismatched. This coarsely filters out observation data pairs that may belong to the same physical target, obtaining a candidate set of correlated targets, thereby significantly reducing the dimensionality of subsequent fine-grained correlation calculations.

[0083] Furthermore, multi-dimensional feature vectors of the candidate associated target set are extracted, and Mahalanobis distance is used to perform fine association calculation based on the multi-dimensional feature vectors to obtain the association matrix.

[0084] Specifically, for each observation data in the candidate associated target set, a multi-dimensional feature vector containing attributes such as position, velocity, acceleration, and direction of motion is extracted; at the same time, the obtained uncertainty measurement data (i.e., the comprehensive error covariance matrix) is imported; the deviation between the multi-dimensional feature vectors is standardized and weighted using the uncertainty measurement data to solve the Mahalanobis distance, so as to accurately measure the feature similarity considering the influence of measurement noise and spatiotemporal error; the Mahalanobis distance or association probability between each heterogeneous observation data pair is filled into a two-dimensional matrix to construct an association matrix reflecting the degree of belonging between multi-source data.

[0085] Furthermore, the Hungarian algorithm or auction algorithm is used to solve the correlation matrix to obtain the globally optimal data correlation result.

[0086] Specifically, to avoid misassociations caused by local greedy matching, this step transforms the data association problem into an assignment problem or a combinatorial optimization problem. The Hungarian algorithm or auction algorithm is used to globally optimize the constructed association matrix. Under the premise of satisfying the "one-to-one" association constraint, the matching combination that minimizes the overall association cost or maximizes the overall similarity is solved to obtain the globally optimal data association result, thereby achieving accurate mapping and assignment of multi-sensor observation data to existing target tracks or new targets.

[0087] Furthermore, based on the globally optimal data association results and uncertainty measurement data, the spatiotemporal aligned data is sequentially fused in chronological order to obtain the multi-source fused target track.

[0088] Specifically, based on the globally optimal data association results, the spatiotemporally aligned data of each sensor belonging to the same target are confirmed; a sequential filtering fusion architecture (such as sequential Kalman filtering or unscented Kalman filtering) is adopted, and the uncertainty measurement data (error covariance matrix) is used as the basis for updating the filter gain and measurement weights; according to the time stamp order of the observation data, the target's state vector (such as position and velocity) is recursively filtered and updated and noise smoothed frame by frame; low-precision observation data is corrected by high-precision observation data to eliminate outlier jitter in single-sensor measurements, and finally, a smooth, continuous and highly accurate multi-source fused target track is output.

[0089] This application's embodiments employ a two-layer association strategy of "coarse association filtering + Mahalanobis distance fine calculation," which significantly reduces the dimensionality of association calculations while considering the distribution of measurement errors, thus significantly improving the data association accuracy under high-density targets. The Hungarian / auction algorithm is used to solve the association matrix, avoiding local greedy matching errors; combined with a sequential filtering architecture, continuous and smooth fusion of multi-source data is achieved, improving track continuity and tracking accuracy.

[0090] In some embodiments, dynamic target group identification and hierarchical aggregation processing are performed on multi-source fusion target tracks to obtain aggregated situational data and target display priority data. This includes: using a density-based clustering algorithm to identify target groups on multi-source fusion target tracks, wherein clustering parameters are adaptively adjusted according to the target distribution density, and the stability of the groups is evaluated in combination with the structural consistency of historical moments to obtain the target group division results.

[0091] Specifically, to avoid information overload caused by the stacking of massive numbers of dense targets on the interface, the system employs density-based clustering algorithms (such as the improved DBSCAN algorithm) to perform real-time clustering analysis on multi-source fusion target tracks. During the clustering process, the system dynamically and adaptively adjusts clustering parameters (such as neighborhood radius and minimum point threshold) based on the local spatial distribution density of targets within the current region, ensuring reasonable grouping for both high-density and sparse regions. Simultaneously, a temporal correlation mechanism is introduced to compare the current clustering results with the group structures from historical moments. By calculating the consistency of motion direction and the degree of relative distance preservation, the structural stability of the groups is evaluated, effectively eliminating frequent disbanding and reconstruction of groups caused by the instantaneous out-of-group behavior or edge jitter of individual targets. Ultimately, the system outputs smooth, continuous, and stable target group partitioning results.

[0092] Furthermore, based on the target group division results, and combined with the viewport scaling ratio and spatial distribution characteristics, the multi-source fusion target tracks are subjected to multi-scale hierarchical aggregation processing to generate group or target representation forms corresponding to the display level, thus obtaining aggregated situational data.

[0093] Specifically, based on the target group segmentation results mentioned above, the system combines the viewport zoom level of the current visualization interface with the spatial distribution range and geometric contours of the target groups to construct a multi-scale hierarchical aggregation model. At a small zoom level (macroscopic, large-scale perspective), multiple tightly clustered target tracks are abstracted and aggregated into a single group node or group contour representation, and statistical vector data containing the number of targets, group type composition, and overall movement trend are appended to the aggregation node. At a magnified viewport, the group nodes are progressively expanded into subgroups or independent individual target tracks according to the resolution; thus generating multi-scale aggregated situational data that adapts to different display levels and resolution requirements.

[0094] Furthermore, based on the target's attribute characteristics, timeliness characteristics, and operator interaction characteristics, a multi-dimensional scoring model is used to calculate and sort the priority of each target and target group, thus obtaining target display priority data.

[0095] Specifically, in order to achieve a reasonable allocation of display resources and operator attention, the system establishes a multi-dimensional scoring model to dynamically prioritize each independent target and target group: Extracting attribute features: including static and dynamic attributes such as the target's motion state (e.g., high speed, high acceleration, or abnormal trajectory) and type importance level; Extracting timeliness features: including the time interval of data updates, information freshness, and confidence of track extrapolation. Targets that are more recent in time and more active in their state are given higher timeliness weights. Extract operator interaction features: capture the operator's current operation behavior, such as mouse focus hover, manual selection of area, highlighting or retrieving selected target / area, and apply a significant interaction weighting factor to the target currently of the operator's attention; Based on the above multidimensional features, the comprehensive priority score of each target and target group is calculated by a preset weighted scoring function, and the scores are sorted in descending order to generate target display priority data to guide subsequent rendering sampling and avoidance control.

[0096] This application's embodiments effectively eliminate the display stacking of dense targets on the interface by employing a density-based adaptive clustering algorithm combined with historical structure consistency assessment, while avoiding frequent disbanding and reconstruction of groups due to transient changes in individual targets. It achieves adaptive presentation of macroscopic groups or microscopic individuals as the viewport zooms in; and combines a multi-dimensional scoring model to accurately assess target priority, ensuring that key targets and focal information are presented preferentially in the operator's line of sight.

[0097] In some embodiments, based on target display priority data and viewport information, the aggregated situation data and multi-source fusion target tracks are dynamically rendered to obtain a multi-dimensional situation visualization interface, including: graphically mapping the spatiotemporal characteristics of the multi-source fusion target tracks to obtain track situation expression data.

[0098] Specifically, the system extracts spatiotemporal features such as three-dimensional spatial position, velocity, heading angle, and historical trajectory from the multi-source fusion target trajectory. Using graphical mapping rules, these continuous spatiotemporal state quantities are converted into visualized graphical primitives. For example, the target's current position is mapped to a vector symbol icon, the historical trajectory is mapped to a trajectory tail line with a time decay effect, the velocity and direction are mapped to dynamic velocity vector lines, and the state prediction is mapped to an ellipse (or grid box) indicating the uncertainty of future position prediction, generating trajectory situational awareness data that intuitively expresses the dynamic motion characteristics of the individual target.

[0099] Furthermore, the situational data and situational elements in the multi-source fusion target trajectory are subjected to multi-dimensional perception attribute fusion visual representation processing to obtain information representation data.

[0100] Specifically, to avoid complex perception attributes occupying too much interface space, the system performs multi-channel visual encoding and representation processing on the multi-dimensional perception attributes (such as sensor measurement confidence, coverage, data source type, and status anomaly alarms) contained in the aggregated situational data and multi-source fusion target tracks. By mapping different perception attributes to different visual channels (such as color saturation, symbol shape, boundary line type, transparency, and dynamic pulse flicker), the system achieves the superposition and fusion of multi-dimensional attribute information in visual presentation (e.g., using transparency to represent the magnitude of uncertainty, and using edge line type to represent the sensor coverage area), generating highly condensed and easily perceptible information representation data.

[0101] Furthermore, the target groups in the aggregated situational data are processed to macroscopically express their group distribution characteristics and overall evolution trends, resulting in group situational expression data.

[0102] Specifically, for target groups in aggregated situational data, the system graphically abstracts and represents their overall distribution geometric characteristics and macroscopic evolution trends: it uses the convex hull algorithm or α-Shape algorithm to extract the outer boundary of the target group and draws the geometric envelope contour representing the spatial range occupied by the group; at the same time, it calculates the average motion velocity of the group's centroid and the overall diffusion / convergence vector, and superimposes macroscopic vector arrows and density variation heatmaps representing the overall evolution trend on the group's geometric contour to generate group situational expression data reflecting the macroscopic situation of the group.

[0103] Furthermore, based on viewport information and target display priority data, multi-scale adaptive rendering control processing is performed on track situation representation data, information representation data, and group situation representation data to obtain a multi-dimensional situation visualization interface.

[0104] Specifically, the system combines current viewport information (including display resolution, map viewport boundaries, zoom level, and viewing angle) with obtained target display priority data, and employs a layered detail and adaptive rendering sampling control strategy: Frustum culling is performed based on the viewport range, and rendering calculations are performed only on primitives within the viewport. By combining the scaling level and the target display priority score, the visibility and rendering granularity of each primitive are dynamically controlled (e.g., high-priority targets are rendered with full flight trails and detailed attribute labels first, while low-priority targets are simplified to point primitives or automatically sampled and hidden from the macro view). The algorithm-based primitive text anti-occlusion and avoidance and adaptive layer overlay drawing are adopted to ensure that high-priority information is always on the top layer and there is no overlap or collision. Through the above multi-scale adaptive rendering control, the allocation of GPU / CPU resources is optimized, and a multi-dimensional situational visualization interface with clear output, no information stacking interference, and high frame rate smooth operation is achieved.

[0105] This application's embodiments map complex spatiotemporal states, confidence levels, and coverage ranges to multi-channel visual language (color, transparency, line type, etc.), achieving intuitive visual presentation under high information density. It can clearly express individual trajectory predictions and intuitively display group evolution trends; employing detail level and adaptive rendering control avoids primitive overlap and occlusion, significantly improving rendering performance and FPS frame rate.

[0106] In some embodiments, the visualization interface quality index and operator interaction behavior data of the multi-dimensional situation visualization interface are obtained, optimization processing is performed based on the visualization interface quality index and operator interaction behavior data to obtain real-time optimization parameters, and the data processing and rendering strategy is dynamically adjusted based on the real-time optimization parameters, including: performing visual presentation effect evaluation processing on the multi-dimensional situation visualization interface to obtain visualization interface quality index.

[0107] Specifically, during the dynamic rendering and presentation of the interface, the system uses a runtime status monitoring module to evaluate the visual presentation effect of the multi-dimensional situational visualization interface in real time. Evaluation metrics include, but are not limited to: real-time rendering frame rate, GPU / CPU resource utilization, interface primitive collision and text overlap, layer rendering latency, and interface visual congestion (such as primitive density per unit area within the viewport). By sampling and comprehensively calculating these parameters in real time, the system assesses the smoothness and visual clarity of the current interface, generating quantitative visualization interface quality metrics that characterize the presentation effect.

[0108] Furthermore, the operator's interface interaction actions are tracked and their intent inferred to obtain operator interaction behavior data.

[0109] Specifically, the system uses an interaction tracking module to capture and record various interactive actions of the operator on the visual interface in real time, including mouse panning, viewport zooming, focus selection, area selection, layer switching, and high-frequency retrieval. Furthermore, it uses intent inference algorithms (such as interaction analysis based on hidden Markov models or sliding time windows) to perform pattern recognition on the operator's continuous interactive behavior, inferring the operator's current focus of attention (such as a specific area, high-risk target group, or specific sensor type) and operational load status, and obtaining operator interaction behavior data that includes interaction type, coordinates of the area of ​​interest, and intent confidence.

[0110] Furthermore, based on the visual interface quality indicators and operator interaction behavior data, comprehensive optimization calculations are performed to obtain real-time optimization parameters.

[0111] Specifically, the system establishes a closed-loop feedback optimization model, using the obtained interface quality indicators and operator interaction behavior data as joint inputs: When a decline in interface quality metrics is detected (such as frame rate falling below a preset threshold or severe occlusion of graphic elements), system performance optimization rules are triggered. Based on the operator interaction behavior data, the focus areas are identified, and non-focus areas are downsampled or simplified. By using a multi-objective optimization algorithm for comprehensive optimization calculations, the real-time optimization parameters that optimize interface quality and interactive response are obtained while ensuring high fidelity of information in the operator's focus area.

[0112] Furthermore, based on real-time optimization parameters, the clustering algorithm parameters, fusion processing accuracy, resource allocation strategy, and interface refresh rate in the data processing stage are dynamically adjusted.

[0113] Specifically, the system feeds back the calculated real-time optimization parameters to the data processing front-end and rendering engine in real time, achieving adaptive dynamic adjustment across the entire process: Adjust clustering algorithm parameters: Adjust the clustering neighborhood radius or minimum number of points threshold in reverse according to the optimization parameters to improve the degree of clustering and reduce the number of independently rendered nodes when the interface is congested or the performance is degraded. Adjust the fusion processing accuracy: Dynamically control the sequential filtering update step size and extrapolation frequency according to the optimization parameters, and appropriately relax the filtering accuracy for targets in non-interested areas to save computing resources. Adjust resource allocation and refresh rate: Adjust the detail level threshold, GPU computing power tilt strategy and viewport refresh rate in the rendering stage according to the optimization parameters, and prioritize high frame rate and high detail rendering of the operator's focus area to achieve adaptive closed-loop collaborative optimization of the entire process of "data processing-rendering presentation-interactive feedback".

[0114] This application embodiment monitors interface quality indicators in real time and infers the operator's interaction intent (focus area), causing the system's computing power and rendering resources to be tilted towards the core area currently of concern to the operator. By dynamically adjusting the clustering threshold, filtering step size, and refresh rate in reverse, while maintaining high fidelity in key areas, it dynamically eliminates interface stuttering and display latency, ensuring an extremely smooth interactive experience even in complex and large-scale situations.

[0115] While this application provides the method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-inventive labor. The order of steps listed in this embodiment is merely one possible execution order among many and does not represent the only execution order. In actual device or client product execution, the methods shown in this embodiment or the accompanying drawings can be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment).

[0116] like Figure 3 As shown in the illustration, this application also provides a visual situation display device 300. The device includes: The acquisition module 301 is used to acquire multi-source heterogeneous sensor data, and to analyze and standardize the multi-source heterogeneous sensor data to obtain standardized observation data.

[0117] The processing module 302 is used to perform time benchmark unification and spatial benchmark unification processing on the standardized observation data to obtain spatiotemporally aligned data and corresponding uncertainty measurement data.

[0118] The processing module 302 is also used to perform multi-level data association and sequential fusion processing on spatiotemporal aligned data based on uncertainty measurement data to obtain multi-source fused target tracks.

[0119] The processing module 302 is also used to perform dynamic target group identification and hierarchical aggregation processing on the multi-source fusion target tracks to obtain aggregated situational data and target display priority data.

[0120] The processing module 302 is also used to dynamically render aggregated situation data and multi-source fusion target tracks based on target display priority data and viewport information to obtain a multi-dimensional situation visualization interface.

[0121] The interaction module 303 is used to acquire the visualization interface quality indicators and operator interaction behavior data of the multi-dimensional situation visualization interface, perform optimization processing based on the visualization interface quality indicators and operator interaction behavior data to obtain real-time optimization parameters, and dynamically adjust the data processing and rendering strategies based on the real-time optimization parameters.

[0122] Some modules in the apparatus described in this application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0123] The apparatus or module described in the above embodiments can be implemented by a computer chip or physical entity, or by a product with a certain function. For ease of description, the above apparatus is described by dividing it into various modules according to their functions. When implementing the embodiments of this application, the functions of each module can be implemented in one or more software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0124] The methods, apparatus, or modules described in this application can be implemented in a computer-readable program code manner. The controller can be implemented in any suitable manner, such as a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of a memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code manner, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included within it for implementing various functions can also be considered as structures within the hardware component. Alternatively, the device used to implement various functions can be viewed as either a software module that implements the method or a structure within a hardware component.

[0125] This application also provides an apparatus, the apparatus comprising: a processor; a memory for storing processor-executable instructions; wherein, when the processor executes the executable instructions, it implements the method described in this application.

[0126] This application also provides a non-volatile computer-readable storage medium storing a computer program or instructions thereon, which, when executed, enables the method described in this application embodiment to be implemented.

[0127] Furthermore, in the various embodiments of the present invention, each functional module can be integrated into a processing module, or each module can exist independently, or two or more modules can be integrated into a single module.

[0128] The aforementioned storage media include, but are not limited to, Random Access Memory (RAM), Read-Only Memory (ROM), Cache, Hard Disk Drive (HDD), or Memory Card. The memory can be used to store computer program instructions.

[0129] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product, or it can be embodied in the process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, mobile terminal, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0130] The various embodiments described in this specification are presented in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on its differences from other embodiments. All or part of this application can be used in numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, mobile communication terminals, multiprocessor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices, etc.

[0131] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of this application.

Claims

1. A method for visualizing situational awareness, characterized in that, include: Acquire multi-source heterogeneous sensor data, and perform analysis and standardization processing on the multi-source heterogeneous sensor data to obtain standardized observation data; The standardized observation data is processed to unify the time reference and the spatial reference, resulting in spatiotemporally aligned data and corresponding uncertainty measurement data. Based on the uncertainty measurement data, the spatiotemporal aligned data is subjected to multi-level data association and sequential fusion processing to obtain multi-source fused target tracks; Dynamic target group identification and hierarchical aggregation processing are performed on the multi-source fusion target trajectory to obtain aggregated situational data and target display priority data; Based on the target display priority data and viewport information, the aggregated situation data and the multi-source fusion target trajectory are dynamically rendered to obtain a multi-dimensional situation visualization interface. The visualization interface quality index and operator interaction behavior data of the multi-dimensional situation visualization interface are obtained. Based on the visualization interface quality index and operator interaction behavior data, optimization processing is performed to obtain real-time optimization parameters. Based on the real-time optimization parameters, the data processing and rendering strategies are dynamically adjusted.

2. The method according to claim 1, characterized in that, The process of acquiring multi-source heterogeneous sensor data, parsing and standardizing the multi-source heterogeneous sensor data to obtain standardized observation data includes: The multi-source heterogeneous sensor data is identified and analyzed to obtain analyzed sensor data. The multi-source heterogeneous sensor data includes at least two of the following: radar, photoelectric, electronic reconnaissance, satellite navigation, communication link, meteorology, and artificial intelligence. The parsed sensor data is subjected to integrity detection processing to obtain valid data; based on the signal-to-noise ratio and detection probability calculation index, the valid data is subjected to data quality assessment processing to obtain data credibility assessment results; Identify and remove outlier data points from the valid data to obtain corrected observation data; Based on the data credibility assessment results, the attribute fields and data types of the corrected observation data are normalized and transformed to obtain standardized observation data.

3. The method according to claim 1, characterized in that, The standardized observation data undergoes time and spatial benchmark unification processing to obtain spatiotemporally aligned data and corresponding uncertainty metric data, including: The standardized observation data are subjected to time transformation processing to obtain a unified time series; Based on the difference in sensor sampling frequency, the unified time series is resampled to obtain time-aligned data, and the interpolation error covariance matrix is ​​calculated synchronously. Transform the local coordinates of each sensor to the geocentric-ground-fixed coordinate system to obtain the initial geocentric-ground-fixed coordinates; A dynamic spatial correction model is constructed based on extended Kalman filtering, which models sensor bias as a state variable and uses multi-sensor cross-validation information to update correction parameters online. Based on the updated correction parameters, the initial geocentric and geofixed coordinates are corrected and transformed to obtain spatially corrected geographic coordinate data. The time-aligned data, the interpolation error covariance matrix, and the spatially corrected geographic coordinate data are merged and integrated to obtain spatiotemporal aligned data and corresponding uncertainty measurement data.

4. The method according to claim 1, characterized in that, Based on the uncertainty measurement data, the spatiotemporal aligned data undergoes multi-level data association and sequential fusion processing to obtain a multi-source fused target trajectory, including: Based on spatial distance and velocity characteristics, the spatiotemporal alignment data is subjected to coarse correlation filtering to obtain a set of candidate correlation targets. Extract the multidimensional feature vectors of the candidate associated target set, and perform fine association calculation based on the Mahalanobis distance using the multidimensional feature vectors to obtain the association matrix; The correlation matrix is ​​solved using the Hungarian algorithm or the auction algorithm to obtain the globally optimal data correlation result; Based on the global optimal data association results and the uncertainty measurement data, the spatiotemporal aligned data is sequentially fused in chronological order to obtain the multi-source fused target track.

5. The method according to claim 1, characterized in that, The process of performing dynamic target group identification and hierarchical aggregation on the multi-source fused target trajectory to obtain aggregated situational data and target display priority data includes: A density-based clustering algorithm is used to identify target groups in the multi-source fusion target tracks. The clustering parameters are adaptively adjusted according to the target distribution density, and the stability of the groups is evaluated by combining the structural consistency at historical moments to obtain the target group division results. Based on the target group division results, the multi-source fusion target tracks are subjected to multi-scale hierarchical aggregation processing in combination with the viewport scaling ratio and spatial distribution characteristics to generate group or target representation forms corresponding to the display level, thereby obtaining aggregated situational data. Based on the target's attribute characteristics, timeliness characteristics, and operator interaction characteristics, a multi-dimensional scoring model is used to calculate and sort the priority of each target and target group, thus obtaining target display priority data.

6. The method according to claim 1, characterized in that, The process of dynamically rendering the aggregated situational awareness data and the multi-source fusion target trajectory based on the target display priority data and viewport information to obtain a multi-dimensional situational awareness visualization interface includes: The spatiotemporal characteristics of the multi-source fusion target trajectory are graphically mapped to obtain trajectory situation representation data; The aggregated situational data and the situational elements in the multi-source fusion target trajectory are subjected to multi-dimensional perception attribute fusion visual representation processing to obtain information representation data; The target groups in the aggregated situational data are processed to macroscopically express their group distribution characteristics and overall evolution trends, resulting in group situational expression data. Based on the viewport information and the target display priority data, multi-scale adaptive rendering control processing is performed on the trajectory situation expression data, the information representation data, and the group situation expression data to obtain a multi-dimensional situation visualization interface.

7. The method according to claim 1, characterized in that, The process of acquiring visualization interface quality indicators and operator interaction behavior data of the multi-dimensional situation visualization interface, performing optimization processing based on the visualization interface quality indicators and operator interaction behavior data to obtain real-time optimization parameters, and dynamically adjusting data processing and rendering strategies based on the real-time optimization parameters includes: The visual presentation effect of the multi-dimensional situation visualization interface is evaluated to obtain the visualization interface quality index. The operator's interface interaction actions are tracked and their intent is inferred to obtain operator interaction behavior data; Based on the quality indicators of the visual interface and the operator interaction behavior data, a comprehensive optimization calculation is performed to obtain real-time optimization parameters. Based on the aforementioned real-time optimization parameters, the clustering algorithm parameters and fusion processing accuracy in the data processing stage, as well as the resource allocation strategy and interface refresh rate in the rendering stage, are dynamically adjusted.

8. A visual situation display device, characterized in that, include: The acquisition module is used to acquire multi-source heterogeneous sensor data, and to parse and standardize the multi-source heterogeneous sensor data to obtain standardized observation data. The processing module is used to perform time benchmark unification and spatial benchmark unification processing on the standardized observation data to obtain spatiotemporally aligned data and corresponding uncertainty measurement data; The processing module is also used to perform multi-level data association and sequential fusion processing on the spatiotemporal alignment data based on the uncertainty measurement data to obtain multi-source fusion target tracks; The processing module is also used to perform dynamic target group identification and hierarchical aggregation processing on the multi-source fusion target trajectory to obtain aggregated situational data and target display priority data; The processing module is also used to dynamically render the aggregated situation data and the multi-source fusion target trajectory based on the target display priority data and viewport information to obtain a multi-dimensional situation visualization interface. The interaction module is used to acquire the visualization interface quality indicators and operator interaction behavior data of the multi-dimensional situation visualization interface, perform optimization processing based on the visualization interface quality indicators and operator interaction behavior data to obtain real-time optimization parameters, and dynamically adjust the data processing and rendering strategies based on the real-time optimization parameters.

9. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images