A space-air three-dimensional situation generation and display platform and a working method thereof

Abstract

The application discloses a space-air three-dimensional situation generation and display platform and a working method thereof, relates to the technical field of computers and three-dimensional geographic information visualization, and the display platform comprises a multi-source data space-time reference registration module, a multi-modal independent semantic feature analysis module, a cross-modal semantic level confidence fusion module, a semantic-three-dimensional automatic instantiation mapping and track deduction module, a computing power self-adaptive rendering scheduling module and an uncertainty three-dimensional visualization expression module. The application can solve the problems of heterogeneous data conflicts, rendering computing power bottlenecks and missing data uncertainty expression.

Description

Technical Field

[0001] This application relates to the fields of computer and 3D geographic information visualization technology, and in particular to a space-air 3D situation generation and display platform and its working method. Background Technology

[0002] With the booming development of the "low-altitude economy," the density of aircraft in urban low-altitude airspace has increased dramatically. To ensure low-altitude safety and prevent unauthorized flights and other non-cooperative targets, building a multi-source sensing network consisting of electro-optical pods (visible and infrared), ground-based surveillance radar, spectrum detection equipment, and ADS-B has become an industry consensus. However, in the complex three-dimensional aerospace environment, existing display and situation generation systems face severe challenges.

[0003] Currently, the aerospace multi-source information display platform mainly adopts the traditional physical overlay or data-level fusion paradigm, which has the following shortcomings: 1) heterogeneous data conflicts and lack of semantic decision-making; 2) 3D situation generation is lagging and highly dependent on manual intervention; 3) high-frequency rendering of massive targets leads to computing power bottlenecks; 4) lack of expression of data uncertainty. Summary of the Invention

[0004] The purpose of this application is to provide a three-dimensional aerospace situation generation and display platform based on semantic-level fusion of multi-source heterogeneous sensors, so as to solve the problems of heterogeneous data conflict, rendering computing power bottleneck and lack of expression of data uncertainty.

[0005] To achieve the above objectives, this application provides the following solution: Firstly, this application provides a three-dimensional aerospace situation generation and display platform, including: The multi-source data spatiotemporal reference registration module is used to: perform multi-source data spatiotemporal reference dynamic registration on the received two-dimensional image pixel stream, and generate a standardized spatial point cloud set in the three-dimensional coordinate system of the Earth ellipsoid. The multimodal independent semantic feature parsing module is used to: extract the initial semantic features of each modality target based on the standardized spatial point cloud set and the multimodal independent semantic feature parsing model, and encapsulate the initial semantic features into local semantic entity objects; The cross-modal semantic-level confidence fusion module is used to: generate a global situational object based on the initial confidence parameters in the local semantic entity object and an improved Dempster-Shafer evidence theory calculation engine; and, upon receiving a refusion request signal, dynamically adjust the conflict factor threshold or sensor weight allocation in the Dempster-Shafer evidence theory calculation engine to regenerate the global situational object; the global situational object consists of globally unique three-dimensional coordinates, motion vectors, fused semantic categories, historical track features, and global fusion confidence. The semantic-3D automatic instantiation mapping and trajectory inference module is used to: trigger the automatic instantiation mapping mechanism when the global fusion confidence is greater than a preset confidence threshold, and obtain the target semantic category, historical trajectory behavior characteristics, and environmental constraints and disturbance factors of the global situation object; based on the target semantic category, historical trajectory behavior characteristics, and environmental constraints and disturbance factors, correct the initial prediction uncertainty generated by the kinematic state transition equation and error covariance matrix, and generate a 3D error prediction trajectory tube with a multi-factor correction adaptive radius; The computing power adaptive rendering scheduling module is used to: extract the category, speed and spatial distance attributes of the global situation object, substitute them into the comprehensive threat index assessment model to calculate the current threat index, and dynamically adjust the rendering multi-detail level and front-end refresh rate of the generated 3D error prediction trajectory tube and its associated field based on the current threat index; The uncertainty 3D visualization expression module is used to: when the global fusion confidence is lower than the certainty threshold and is in the suspected interval, shield the rigid body model mapping, trigger the uncertainty 3D visualization expression mechanism, generate a gradient probability point cloud based on a 3D Gaussian distribution that is inversely correlated with the global fusion confidence value, and perform non-rigid body rendering; and when the duration of the global fusion confidence being in the suspected interval is greater than the preset feedback trigger duration, generate a refusion request signal and feed the refusion request signal back to the cross-modal semantic level confidence fusion module.

[0006] Secondly, this application provides a working method for a three-dimensional aerospace situation generation and display platform, including: The multi-source data spatiotemporal reference registration module performs multi-source data spatiotemporal reference dynamic registration on the received two-dimensional image pixel stream to generate a standardized spatial point cloud set in the three-dimensional coordinate system of the Earth ellipsoid. The multimodal independent semantic feature parsing module extracts the initial semantic features of each modality target based on the standardized spatial point cloud set and the multimodal independent semantic feature parsing model, and encapsulates the initial semantic features into local semantic entity objects. The cross-modal semantic-level confidence fusion module generates a global situational object based on the initial confidence parameters in the local semantic entity object and an improved Dempster-Shafer evidence theory calculation engine. The global situational object consists of globally unique three-dimensional coordinates, motion vectors, fused semantic categories, historical track features, and global fusion confidence. When the global fusion confidence level is greater than the preset confidence threshold, the automatic instantiation mapping mechanism is triggered through the semantic-3D automatic instantiation mapping and trajectory inference module to generate a 3D error prediction trajectory tube with multi-factor correction adaptive radius. The computing power adaptive rendering scheduling module extracts the category, speed and spatial distance attributes of the global situational object, substitutes them into the comprehensive threat index assessment model to calculate the current threat index, and dynamically adjusts the rendering multi-detail level and front-end refresh rate of the generated 3D error prediction trajectory tube and its associated field based on the current threat index. Based on the three-dimensional error prediction trajectory tube, the future spatial position of the target within the preset prediction time step is predicted to obtain the predicted threat index. When the predicted threat index reaches a threat range higher than the current display level, or when the current threat index shows an upward trend in multiple consecutive sampling periods and the increase is greater than the preset increase threshold, the corresponding three-dimensional model, texture material, shader program, alarm effect or rendering cache resources are preloaded in advance. When the global fusion confidence is lower than the certainty threshold and is in the suspected range, the rigid body model mapping is shielded by the uncertainty 3D visualization expression module, triggering the uncertainty 3D visualization expression mechanism. The gradient probability point cloud, which is inversely correlated with the global fusion confidence value, is generated based on the 3D Gaussian distribution for non-rigid body rendering. When the duration of the global fusion confidence score being in the suspected range exceeds the preset feedback trigger duration, a refusion request signal is generated through the uncertainty 3D visualization expression module, and the refusion request signal is fed back to the cross-modal semantic level confidence fusion module. The cross-modal semantic level confidence fusion module dynamically adjusts the conflict factor threshold or sensor weight allocation in the Dempster-Shafer evidence theory calculation engine according to the refusion request signal, and recalculates the global fusion confidence score based on the adjusted fusion parameters.

[0007] According to the specific embodiments provided in this application, this application has the following technical effects: This application provides a three-dimensional aerospace situation generation and display platform and its working method. It generates a three-dimensional error prediction trajectory tube with a multi-factor correction adaptive radius through a semantic-3D automatic instantiation mapping and trajectory inference module, solving the problem of intuitively quantifying trajectory prediction errors in traditional situation display. This achieves dynamic and accurate visualization of the uncertainty of target motion trajectories. A computing power adaptive rendering scheduling module adjusts the rendering of multiple detail levels and the front-end refresh rate based on the threat index. Combined with an uncertainty 3D visualization expression module, it triggers non-rigid rendering of gradient probability point clouds when the global fusion confidence level is in the suspected range. This resolves the contradiction between the accuracy of situation information display and resource consumption under different confidence levels, achieving intelligent visualization scheduling that prioritizes fine rendering of high-threat targets and probabilistic expression of low-confidence targets, effectively improving the reliability and efficiency of aerospace 3D situation display. Furthermore, by establishing a re-fusion feedback mechanism between the uncertainty 3D visualization expression module and the cross-modal semantic-level confidence fusion module, targets that remain in the suspected range for a long time can trigger adaptive adjustment of fusion parameters and recalculation of confidence, avoiding the fusion results remaining in an uncertain state for a long time, and improving the convergence of multi-source fusion results and the decision-making effectiveness of situation display. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 The module architecture and data flow diagram of the aerospace 3D situation generation and display platform based on semantic-level fusion of multi-source heterogeneous sensors provided in this application; Figure 2 The internal logic diagram of the cross-modal semantic-level confidence fusion module provided in this application; Figure 3 A schematic diagram of the working method of a three-dimensional aerospace situation generation and display platform provided in this application; Figure 4 This is a schematic diagram of a computer device structure provided for this application. Detailed Implementation

[0010] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0011] Currently, aerospace multi-source information display platforms mainly adopt traditional physical overlay or data-level fusion paradigms, and their main shortcomings include: 1) Heterogeneous data conflicts and lack of semantic decision-making: Traditional systems simply overlay radar points and optical video streams onto two-dimensional or three-dimensional maps. When sensors conflict due to occlusion or interference, the system cannot perform cross-modal cross-validation, resulting in high false alarm and missed alarm rates, and making it difficult to form decision-making criteria with definite semantic labels.

[0012] 2) Lagging 3D situation generation and high reliance on manual intervention: The existing platform's 3D defense zones, threat radiation circles, and flight trajectories largely rely on manual presets or static configurations. When facing high-speed, highly maneuverable low-altitude non-cooperative targets, the system cannot automatically map and generate matching 3D situation primitives based on the target's real-time semantic attributes, resulting in delayed perception for command personnel.

[0013] 3) High-frequency rendering of massive targets leads to computing power bottleneck: In low-altitude large-scale cluster scenarios, traditional 3D GIS engines use the same high-precision, high-frequency real-time rendering for all targets, resulting in severe lag and a sharp drop in frame rate on the front-end display platform. There is a lack of an adaptive rendering computing power scheduling mechanism based on changes in the semantic state of the targets.

[0014] 4) Lack of expression of data uncertainty: Sensor detection often has blind spots and errors, but existing platforms tend to use deterministic rigid body models to display targets, which cannot intuitively present the "confidence / uncertainty" output by the underlying fusion algorithm, which can easily mislead command and decision-making.

[0015] Therefore, the purpose of this application is to provide a three-dimensional aerospace situation generation and display platform based on semantic-level fusion of multi-source heterogeneous sensors, so as to solve the problems of heterogeneous data conflict, rendering computing power bottleneck and lack of expression of data uncertainty.

[0016] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0017] Example 1 like Figure 1 As shown, this embodiment provides a three-dimensional aerospace situation generation and display platform, including: The multi-source data spatiotemporal reference registration module is used to: perform multi-source data spatiotemporal reference dynamic registration on the received two-dimensional image pixel stream, and generate a standardized spatial point cloud set in the three-dimensional coordinate system of the Earth ellipsoid. The multimodal independent semantic feature parsing module is used to: extract the initial semantic features of each modality target based on the standardized spatial point cloud set and the multimodal independent semantic feature parsing model, and encapsulate the initial semantic features into local semantic entity objects; The cross-modal semantic-level confidence fusion module is used to: generate a global situational object based on the initial confidence parameters in the local semantic entity object and an improved Dempster-Shafer evidence theory calculation engine; and, upon receiving a refusion request signal, dynamically adjust the conflict factor threshold or sensor weight allocation in the Dempster-Shafer evidence theory calculation engine to regenerate the global situational object; the global situational object consists of globally unique three-dimensional coordinates, motion vectors, fused semantic categories, historical track features, and global fusion confidence. The semantic-3D automatic instantiation mapping and trajectory inference module is used to: trigger the automatic instantiation mapping mechanism when the global fusion confidence is greater than a preset confidence threshold, and obtain the target semantic category, historical trajectory behavior characteristics, and environmental constraints and disturbance factors of the global situation object; based on the target semantic category, historical trajectory behavior characteristics, and environmental constraints and disturbance factors, correct the initial prediction uncertainty generated by the kinematic state transition equation and error covariance matrix, and generate a 3D error prediction trajectory tube with a multi-factor correction adaptive radius; The computing power adaptive rendering scheduling module is used to: extract the category, speed and spatial distance attributes of the global situation object, substitute them into the comprehensive threat index assessment model to calculate the current threat index, and dynamically adjust the rendering multi-detail level and front-end refresh rate of the generated 3D error prediction trajectory tube and its associated field based on the current threat index; The uncertainty 3D visualization expression module is used to: when the global fusion confidence is lower than the certainty threshold and is in the suspected interval, shield the rigid body model mapping, trigger the uncertainty 3D visualization expression mechanism, generate a gradient probability point cloud based on a 3D Gaussian distribution that is inversely correlated with the global fusion confidence value, and perform non-rigid body rendering; and when the duration of the global fusion confidence being in the suspected interval is greater than the preset feedback trigger duration, generate a refusion request signal and feed the refusion request signal back to the cross-modal semantic level confidence fusion module.

[0018] In some embodiments, the display platform further includes a 3D asset library storage module, which is used to pre-store low-face-count geometric models, standard texture models and high-precision physical material models corresponding to different semantic categories, so that the semantic-3D automatic instantiation mapping and trajectory inference module can schedule them as needed.

[0019] The platform adopts an edge-cloud collaborative deployment architecture: the multi-source data spatiotemporal benchmark registration module, the multimodal independent semantic feature parsing module, and the cross-modal semantic-level confidence fusion module are deployed on edge computing nodes, while the semantic-3D automatic instantiation mapping and trajectory inference module, the computing power adaptive rendering scheduling module, and the uncertainty 3D visualization expression module are deployed on cloud rendering servers.

[0020] In some embodiments, the computing power adaptive rendering scheduling module is further configured to: predict the future spatial position of the target within a preset prediction time step based on the 3D error prediction trajectory tube, obtain a predicted threat index, and preload the multi-level detail 3D model, texture material, shader program, alarm effect or rendering cache resources corresponding to the predicted threat index in advance when the predicted threat index is higher than the threat range corresponding to the current display level, or when the current threat index shows an upward trend in multiple consecutive sampling periods and the increase is greater than a preset increase threshold.

[0021] Example 2 like Figure 1 and Figure 3 As shown in the figure, this embodiment provides a working method for a three-dimensional aerospace situation generation and display platform, including: Step 301: Through the multi-source data spatiotemporal reference registration module, the received two-dimensional image pixel stream is dynamically registered using multi-source data spatiotemporal reference to generate a standardized spatial point cloud set in the three-dimensional coordinate system of the Earth ellipsoid. Step 302: The multimodal independent semantic feature parsing module extracts the initial semantic features of each modality target based on the standardized spatial point cloud set and the multimodal independent semantic feature parsing model, and encapsulates the initial semantic features into local semantic entity objects; Step 303: The cross-modal semantic level confidence fusion module generates a global situational object based on the initial confidence parameters in the local semantic entity object and the improved Dempster-Shafer evidence theory calculation engine. The global situational object consists of globally unique three-dimensional coordinates, motion vectors, fused semantic categories, historical track features, and global fusion confidence. Step 304: When the global fusion confidence is greater than the preset confidence threshold, the automatic instantiation mapping mechanism is triggered through the semantic-3D automatic instantiation mapping and trajectory inference module to generate a 3D error prediction trajectory tube with multi-factor correction adaptive radius; Step 305: Extract the category, speed and spatial distance attributes of the global situation object through the computing power adaptive rendering scheduling module, substitute them into the comprehensive threat index evaluation model to calculate the current threat index, and dynamically adjust the rendering multi-detail level and front-end refresh rate of the generated 3D error prediction trajectory tube and its associated field based on the current threat index. Step 306: Based on the three-dimensional error prediction trajectory tube, predict the future spatial position of the target within the preset prediction time step to obtain the predicted threat index. When the predicted threat index reaches a higher threat range than the current display level, or when the current threat index shows an upward trend in multiple consecutive sampling periods and the increase is greater than the preset increase threshold, preload the corresponding three-dimensional model, texture material, shader program, alarm effect or rendering cache resources in advance. Step 307: When the global fusion confidence is lower than the certainty threshold and is in the suspected range, the rigid body model mapping is shielded through the uncertainty 3D visualization expression module, triggering the uncertainty 3D visualization expression mechanism, and generating a gradient probability point cloud based on the 3D Gaussian distribution that is inversely correlated with the global fusion confidence value for non-rigid body rendering. Step 308: When the duration of the global fusion confidence score being in the suspected interval is greater than the preset feedback trigger duration, a refusion request signal is generated through the uncertainty 3D visualization expression module, and the refusion request signal is fed back to the cross-modal semantic level confidence fusion module; the cross-modal semantic level confidence fusion module dynamically adjusts the conflict factor threshold or sensor weight allocation in the Dempster-Shafer evidence theory calculation engine according to the refusion request signal, and recalculates the global fusion confidence score based on the adjusted fusion parameters.

[0022] In one exemplary embodiment, when performing steps 301-308, the specific steps may be as follows: Step 1: The multi-source data spatiotemporal reference registration module receives raw radar polar coordinate points at different sampling rates and two-dimensional image pixel streams from the photoelectric camera, performs dynamic registration of the multi-source data spatiotemporal reference, and aligns the heterogeneous data to the reference timestamp. It is then solved into a standardized spatial point cloud set in the WGS-84 Earth ellipsoid three-dimensional coordinate system; Specifically, the process first acquires the original polar coordinate points of the radar at different sampling rates and the two-dimensional image pixel streams of the photoelectric camera, then performs dynamic registration of the multi-source data spatiotemporal reference, aligning the heterogeneous data to the reference timestamp. It is then solved into a standardized spatial point cloud set in the WGS-84 Earth ellipsoid three-dimensional coordinate system.

[0023] For photoelectric detection data, the three-dimensional spatial coordinate matrix The solution obtained by inverse transformation of perspective projection is as follows: ; in, This is the intrinsic parameter matrix of the photoelectric camera, which includes the camera's focal length and optical center offset parameters; The rotation matrix provides real-time feedback from the gimbal. Let be the translation vector of the photoelectric camera; and These are the inverse matrices of the corresponding matrices.

[0024] For the original polar coordinate points of the radar, obtain the radial distance R and azimuth angle in the polar coordinate system. and pitch angle The three-dimensional spatial coordinate matrix is ​​obtained by rotation and translation in the coordinate system. : ; in, This is the geodetic reference coordinate vector of the radar equipment itself in the WGS-84 coordinate system.

[0025] Specifically, in this embodiment, the parameter settings and spatial transformation derivation are as follows: The system is set to acquire a polar target captured by radar at a certain moment, with a radial distance R = 500m and an azimuth angle of [missing information]. = Pitch angle = Simultaneously, the photoelectric camera captures the target's image pixels (u=960, v=540). Through matrix inversion and affine transformation using the camera's intrinsic and extrinsic parameter matrices, the two-dimensional pixel coordinates, combined with depth information, are transformed into the world coordinate system. The two are then precisely aligned to a unified three-dimensional grid coordinate system (102.5, 340.2, 150.0).

[0026] Step 2: The multimodal independent semantic feature parsing module deploys a multimodal independent semantic feature parsing model on the standardized spatial point cloud set generated in Step 1, extracts the initial semantic features of each modality target, and encapsulates them in a structured manner as local semantic entity objects. ; Local semantic entity objects For a structure containing multidimensional parameters, its complete definition is: ; in, Let be the local track number within the i-th sensor; For the unified timestamp after registration; It is a three-dimensional coordinate matrix; It is a velocity vector; A set of candidate semantic categories under the system identification framework; This is the initial confidence vector (i.e., basic probability assignment) belonging to each candidate category.

[0027] In this embodiment, each front-end sensor outputs an independent detection probability: The radar branch, after being processed by CFAR and a pre-classifier, outputs an initial confidence vector. ={UAV:0.7,Bird:0.1,Unknown:0.2}.

[0028] The optical branch, after being processed by the visual target detection network, suffers from blurred feature extraction due to the influence of low-altitude fog, resulting in an initial confidence vector output. ={UAV:0.4,Bird:0.4,Unknown:0.2}. At this point, the system has not yet reached a unified conclusion.

[0029] Step 3: As Figure 2 As shown, the local semantic entity object constructed by the cross-modal semantic level confidence fusion module based on step 2 The initial confidence parameters are extracted and substituted into an improved Dempster-Shafer (DS) evidence theory calculation engine for conflict resolution and orthogonal calculation, generating a result containing globally unique three-dimensional coordinates, motion vectors, and a globally fused confidence score. The global situational object.

[0030] Specifically, the cross-modal semantic-level confidence fusion module is also used to receive the refusion request signal fed back by the uncertainty 3D visualization expression module; when the refusion request signal is received, the cross-modal semantic-level confidence fusion module dynamically adjusts the conflict factor threshold or sensor weight allocation in the Dempster-Shafer evidence theory calculation engine, and recalculates the global fusion confidence based on the adjusted fusion parameters, so that targets that have been in the suspected range for a long time can re-enter the fusion judgment process.

[0031] Specifically, extract the confidence vector of the optical sensor (let's call it...). ) and the confidence vector of the radar sensor (let's call it) First, calculate the conflict factor K, which characterizes the degree of discrepancy in observations from multiple sensors: ; Here, B and C represent two independent and non-overlapping sets of semantic categories inferred from optical and radar sensors, respectively. Subsequently, conflict resolution is achieved through DS orthogonality and formulas, and the global fusion confidence score of the target belonging to a specific category A is calculated. Encapsulate it into a global situational awareness object and pass it to the next stage: ; In this embodiment, the confidence vectors of radar and optics are substituted: First, calculate the conflict factor K between the two. (UAV) (Bird)+ (Bird) (UAV)=0.7 0.4 + 0.1 0.4 = 0.32.

[0032] The global fusion confidence level for the target "unmanned aerial vehicle (UAV)" was then calculated. =(0.7 0.4 + 0.7 0.2+0.2 0.4) / (1-0.32) 0.735.

[0033] System set confidence threshold =0.8, suspected threshold =0.4. Since 0.735 < 0.8, the target is currently classified as a suspected target.

[0034] To facilitate the implementation of the engineering system, the core code logic for cross-modal confidence fusion based on DS evidence theory in this step is shown in Table 1: Table 1 Code Logic Table

[0035] Step 4: The global fusion confidence score obtained by the semantic-3D automatic instantiation mapping and trajectory inference module in step 3. Greater than the preset confidence threshold When the "semantic-3D" ​​automatic instantiation mapping mechanism is triggered, a matching high-precision 3D model is scheduled from the 3D asset library, and a 3D error prediction trajectory tube with a multi-factor correction adaptive radius is dynamically generated by using Kalman filtering, error covariance matrix and multi-factor uncertainty correction. The spatial morphology of the 3D error prediction trajectory tube is jointly constrained by the target's kinematic characteristics and multi-sensor covariance: extracting the current 3D coordinates of the global situational object. Velocity vector acceleration vector The next time step is calculated by solving the state transition equation. The center skeleton coordinates of the trajectory tube : ; Simultaneously, the error covariance matrix of the multi-sensor joint solution system is introduced. Extract its largest eigenvalue Combining multi-factor uncertainty correction, the dynamic calculation of time variation with prediction time is performed. Diffusion trajectory tube cross-sectional radius : ; + + ; in, This is the preset confidence interval scaling factor. This is a correction coefficient for multi-factor uncertainty. For the category mobility uncertainty factor corresponding to the target semantic category, Deviation factor of historical behavior pattern corresponding to the target historical track. This refers to the environmental constraints and disturbance uncertainties corresponding to the current airspace environment. The normalized weighting coefficients sum to 1; this logarithmic diffusion formula makes the generated 3D trajectory tube visually appear as a funnel-shaped spatial bounding box that smoothly expands with the accumulation of errors over time.

[0036] In this embodiment, the parameters are set as follows: the confidence interval scaling factor is set to... =1.5. As the fog gradually dissipates and the optical confidence level recovers, the calculated... The value jumped to 0.95, breaking the confidence threshold. The system instantiated a high-precision 3D UAV model from the asset library. Simultaneously, based on the kinematic equations, it extrapolated the spatial central skeleton and extracted the largest eigenvalue of the covariance matrix. Simultaneously, the system calculates multi-factor uncertainty correction coefficients based on the target semantic category, the degree of deviation from historical behavior patterns, and the constraints of the current airspace environment. The correction factor is then used to correct the cross-sectional radius of the trajectory tube, which varies with the prediction time. A funnel-shaped, semi-transparent error prediction tube with logarithmic diffusion is rendered in real time.

[0037] Step 5: The computing power adaptive rendering scheduling module extracts the category, velocity and spatial distance attributes of the global situational objects from Step 3 and Step 4, substitutes them into the comprehensive threat index assessment model for calculation, and dynamically adjusts the rendering level of detail (LOD) and front-end refresh rate of the 3D target and its associated field generated in Step 4 based on the calculation results.

[0038] Specifically, adaptive 3D rendering scheduling depends on the comprehensive threat index. Real-time calculation: Extracting the fixed threat base corresponding to the fusion category of the global situational awareness object. Euclidean distance between the target and the boundary of the core no-fly zone and target radial velocity Substitute into the following formula to calculate the comprehensive threat index. : ; in, , , These are normalized weight coefficients that add up to 1. The distance attenuation constant; This refers to the speed sensitivity coefficient. This is a nonlinear penalized activation function constructed for the target radial velocity.

[0039] Furthermore, based on the three-dimensional error prediction trajectory generated in step 4, the system obtains the target's future spatial position within a preset prediction time step, and calculates the predicted spatial distance between the future spatial position and the boundary of the core no-fly zone. Simultaneously obtain the predicted radial velocity ,Will and Substituting into the above comprehensive threat index assessment model, the predicted threat index is obtained. .

[0040] When the predicted threat index Reaching a threat level higher than the current display level, or the current threat index. When the rendering index shows an upward trend over multiple consecutive sampling periods and the increase exceeds a preset threshold, the adaptive rendering scheduling module preloads the 3D models, textures, shader programs, alarm effects, or rendering cache resources required for the next rendering multi-level of detail (MLD) or the rendering multi-level of detail corresponding to the predicted threat index. This preloading is only for preparing rendering resources in advance and does not change the currently displayed MLD; the currently displayed MLD is still determined by the current threat index. The corresponding threat range has been determined.

[0041] Based on the calculation Dynamically triggering different actual rendering levels: when < At this time, the control 3D platform is downgraded to LOD-0 mode, loading only low-polygon geometry models, and the trajectory refresh rate is reduced. 1Hz; when < At that time, the control 3D platform is switched to LOD-1 mode, the standard texture model is loaded, and the trajectory refresh rate is set to 10Hz; when At that time, a semantic state mutation interruption occurred, the control 3D platform was upgraded to LOD-2 mode, the camera perspective was forced to track the target, a high-precision physical material model and peripheral radar alarm dome effect were loaded, and the rendering refresh rate was increased to 60Hz.

[0042] In this embodiment, the weights and parameters are set as follows: Category weight =0.3, distance weight =0.5, velocity weight =0.2, distance attenuation constant =0.01, velocity sensitivity coefficient .

[0043] When the target suddenly accelerates and approaches the core no-fly zone, the system first calculates the target's future spatial position within a preset prediction time step based on the three-dimensional error prediction trajectory tube, and then assigns the spatial distance attribute corresponding to the predicted position. and predicted radial velocity Substituting into the comprehensive threat index assessment model, the predicted threat index is obtained. When predicting the threat index When the threat level reaches a range higher than the current display level, the system preloads the high-precision PBR physical material model, peripheral radar alarm dome effects, shader programs, and related rendering cache resources required for LOD-2 level; however, the current display level is still based on the current threat index. Sure.

[0044] Subsequently, when the spatial distance of the target sudden decrease, radial velocity A surge. Substituting into the above formula, we get... =0.88. Because 0.88 is greater than the system's set high-risk threshold. The system immediately triggers a state change interrupt command, switching the rendering level of the target to LOD-2: forcing the rendering engine to track the camera's viewpoint, loading the highest precision PBR physical material model, overlaying the radar alarm dome effect, and forcibly increasing the local refresh rate to 60Hz to ensure smooth monitoring.

[0045] The core code logic for dynamic computing power allocation and rendering level scheduling in this module is shown in Table 2: Table 2 Core Code Logic Table a

[0046] Step 6: Global fusion confidence score obtained from Step 3 by the uncertainty 3D visualization expression module. Below the confidence threshold When the location is within the suspected interval, the rigid body model mapping in step 4 is masked, triggering the uncertainty 3D visualization expression mechanism, which generates a representation based on a 3D Gaussian distribution. Numerical inverse correlation gradient probability point cloud is used for non-rigid body rendering.

[0047] When the global fusion confidence score remains in the suspected range for a duration longer than the preset feedback trigger duration, the uncertainty 3D visualization expression module generates a refusion request signal and feeds the refusion request signal back to the cross-modal semantic level confidence fusion module, causing the cross-modal semantic level confidence fusion module to re-execute the confidence fusion calculation.

[0048] Specifically, when When the value is below the confidence threshold, the rigid body model mapping is disabled, a gradient probability point cloud is generated based on a 3D Gaussian distribution, and the pulse refresh frequency of the internal luminescent particles is calculated. : ; in, This is the frequency adjustment constant. To prevent the denominator from being zero for extremely small positive numbers, a warning visual interaction effect is achieved where the lower the fusion confidence level, the higher the flashing frequency of the target's 3D probability cloud.

[0049] In this embodiment, due to the confidence level initially calculated in step 3 =0.735, which is within the suspected range of [0.4, 0.8). At this point, the engine does not render a concrete drone entity model, but instead calculates the opacity of each voxel based on a 3D Gaussian function to render a set of volumetric probability clouds.

[0050] Set frequency adjustment constant Minimum value Substituting the values ​​into the solution yields the scintillation frequency of the probability cloud. 2.7Hz. The fragment shader inside the engine uses this frequency parameter to control the sinusoidal pulsation of the luminous particles, achieving an uncertain warning interaction effect where "the lower the confidence level, the more rapid the visual flicker."

[0051] The core code logic of this mechanism, which converts the uncertainty of the backend algorithm output into frontend rendering parameters, is shown in Table 3. Table 3 Core Code Logic Table b

[0052] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 4As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores configuration data, sensor observation data, fusion result data, 3D model data, rendering parameter data, and various intermediate data used in the comprehensive threat index calculation process related to the aerospace 3D situation generation and display platform. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a working method for the aerospace 3D situation generation and display platform.

[0053] Those skilled in the art will understand that Figure 4 The structures shown are merely block diagrams of some structures related to the present application and do not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements. In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0054] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0055] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0056] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of the relevant data are carried out in compliance with the relevant data protection laws and policies of the country where the location is located, and with the authorization granted by the owner of the corresponding device.

[0057] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0058] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0059] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0060] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A three-dimensional aerospace situation generation and display platform, characterized in that, include: The multi-source data spatiotemporal reference registration module is used to: perform multi-source data spatiotemporal reference dynamic registration on the received two-dimensional image pixel stream, and generate a standardized spatial point cloud set in the three-dimensional coordinate system of the Earth ellipsoid. The multimodal independent semantic feature parsing module is used to: extract the initial semantic features of each modality target based on the standardized spatial point cloud set and the multimodal independent semantic feature parsing model, and encapsulate the initial semantic features into local semantic entity objects; The cross-modal semantic-level confidence fusion module is used to: generate a global situational object based on the initial confidence parameters in the local semantic entity object and an improved Dempster-Shafer evidence theory calculation engine; and, upon receiving a refusion request signal, dynamically adjust the conflict factor threshold or sensor weight allocation in the Dempster-Shafer evidence theory calculation engine to regenerate the global situational object; the global situational object consists of globally unique three-dimensional coordinates, motion vectors, fused semantic categories, historical track features, and global fusion confidence. The semantic-3D automatic instantiation mapping and trajectory inference module is used to: trigger the automatic instantiation mapping mechanism when the global fusion confidence is greater than a preset confidence threshold, and obtain the target semantic category, historical trajectory behavior characteristics, and environmental constraints and disturbance factors of the global situation object; based on the target semantic category, historical trajectory behavior characteristics, and environmental constraints and disturbance factors, correct the initial prediction uncertainty generated by the kinematic state transition equation and error covariance matrix, and generate a 3D error prediction trajectory tube with a multi-factor correction adaptive radius; The computing power adaptive rendering scheduling module is used to: extract the category, speed and spatial distance attributes of the global situation object, substitute them into the comprehensive threat index assessment model to calculate the current threat index, and dynamically adjust the rendering multi-detail level and front-end refresh rate of the generated 3D error prediction trajectory tube and its associated field based on the current threat index; The uncertainty 3D visualization expression module is used to: when the global fusion confidence is lower than the certainty threshold and is in the suspected interval, shield the rigid body model mapping, trigger the uncertainty 3D visualization expression mechanism, generate a gradient probability point cloud based on a 3D Gaussian distribution that is inversely correlated with the global fusion confidence value, and perform non-rigid body rendering; and when the duration of the global fusion confidence being in the suspected interval is greater than the preset feedback trigger duration, generate a refusion request signal and feed the refusion request signal back to the cross-modal semantic level confidence fusion module.

2. The aerospace three-dimensional situation generation and display platform according to claim 1, characterized in that, Also includes: The 3D asset library storage module is used to pre-store low-face-count geometric models, standard texture models, and high-precision physical material models corresponding to different semantic categories, so that the semantic-3D automatic instantiation mapping and trajectory inference module can schedule them as needed.

3. The aerospace three-dimensional situation generation and display platform according to claim 1, characterized in that, The computing power adaptive rendering scheduling module is also used to: predict the future spatial position of the target within a preset prediction time step based on the 3D error prediction trajectory tube, obtain the predicted threat index, and when the predicted threat index is higher than the threat range corresponding to the current display level, or when the current threat index shows an upward trend in multiple consecutive sampling periods and the increase is greater than a preset increase threshold, preload the multi-detail level 3D model, texture material, shader program, alarm effect or rendering cache resources corresponding to the predicted threat index in advance.

4. A working method for a three-dimensional aerospace situation generation and display platform, characterized in that, include: The multi-source data spatiotemporal reference registration module performs multi-source data spatiotemporal reference dynamic registration on the received two-dimensional image pixel stream to generate a standardized spatial point cloud set in the three-dimensional coordinate system of the Earth ellipsoid. The multimodal independent semantic feature parsing module extracts the initial semantic features of each modality target based on the standardized spatial point cloud set and the multimodal independent semantic feature parsing model, and encapsulates the initial semantic features into local semantic entity objects. The cross-modal semantic-level confidence fusion module generates a global situational object based on the initial confidence parameters in the local semantic entity object and an improved Dempster-Shafer evidence theory calculation engine. The global situational object consists of globally unique three-dimensional coordinates, motion vectors, fused semantic categories, historical track features, and global fusion confidence. When the global fusion confidence level is greater than the preset confidence threshold, the automatic instantiation mapping mechanism is triggered through the semantic-3D automatic instantiation mapping and trajectory inference module to generate a 3D error prediction trajectory tube with multi-factor correction adaptive radius. The computing power adaptive rendering scheduling module extracts the category, speed and spatial distance attributes of the global situational object, substitutes them into the comprehensive threat index assessment model to calculate the current threat index, and dynamically adjusts the rendering multi-detail level and front-end refresh rate of the generated 3D error prediction trajectory tube and its associated field based on the current threat index. Based on the three-dimensional error prediction trajectory tube, the future spatial position of the target within the preset prediction time step is predicted to obtain the predicted threat index. When the predicted threat index reaches a threat range higher than the current display level, or when the current threat index shows an upward trend in multiple consecutive sampling periods and the increase is greater than the preset increase threshold, the corresponding three-dimensional model, texture material, shader program, alarm effect or rendering cache resources are preloaded in advance. When the global fusion confidence is lower than the certainty threshold and is in the suspected range, the rigid body model mapping is shielded by the uncertainty 3D visualization expression module, triggering the uncertainty 3D visualization expression mechanism. The gradient probability point cloud, which is inversely correlated with the global fusion confidence value, is generated based on the 3D Gaussian distribution for non-rigid body rendering. When the duration of the global fusion confidence score being in the suspected range exceeds the preset feedback trigger duration, a refusion request signal is generated through the uncertainty 3D visualization expression module, and the refusion request signal is fed back to the cross-modal semantic level confidence fusion module. The cross-modal semantic level confidence fusion module dynamically adjusts the conflict factor threshold or sensor weight allocation in the Dempster-Shafer evidence theory calculation engine according to the refusion request signal, and recalculates the global fusion confidence score based on the adjusted fusion parameters.

5. The working method of the aerospace three-dimensional situation generation and display platform according to claim 4, characterized in that, Dynamic registration of the received 2D image pixel stream using a multi-source data spatiotemporal reference is performed, specifically including: Acquire the original polar coordinate point traces of radar and the pixel stream of two-dimensional images from an optoelectronic camera at different sampling rates; For a two-dimensional image pixel stream from an electro-optical camera, obtain the two-dimensional pixel coordinates (u,v) and the target depth guided by radar cross-section. By solving the inverse transformation of perspective projection, a unified three-dimensional spatial coordinate matrix is ​​obtained. : ； in, This is the intrinsic parameter matrix of the photoelectric camera; The rotation matrix provides real-time feedback from the gimbal; Let be the translation vector of the photoelectric camera; and These are the inverse matrices of the corresponding matrices; For the original polar coordinate points of the radar, obtain the radial distance R and azimuth angle in the polar coordinate system. and pitch angle The three-dimensional spatial coordinate matrix is ​​obtained by rotation and translation in the coordinate system. : ； in, This is the geodetic reference coordinate vector of the radar equipment itself in the WGS-84 coordinate system.

6. The working method of the aerospace three-dimensional situation generation and display platform according to claim 4, characterized in that, The local semantic entity object For a structure containing multidimensional parameters, the formula expression for the local semantic entity object is: ； in, Let be the local track number within the i-th sensor; For the unified timestamp after registration; It is a three-dimensional coordinate matrix; It is a velocity vector; A set of candidate semantic categories under the system identification framework; This represents the initial confidence vector belonging to each candidate category.

7. The working method of the aerospace three-dimensional situation generation and display platform according to claim 4, characterized in that, Upon receiving a refusion request signal, the evidence weights corresponding to the optical and radar sensors are adjusted based on the refusion request signal, or the conflict factor threshold is adjusted, and the global fusion confidence is recalculated based on the adjusted evidence weights or conflict factor threshold; the formula for calculating the global fusion confidence is as follows: ； Where B and C represent two independent and non-overlapping sets of semantic categories inferred from optical and radar sensors, respectively, and A is any definite category. This represents the global fusion confidence level.

8. The working method of the aerospace three-dimensional situation generation and display platform according to claim 4, characterized in that, The method for generating the three-dimensional error prediction trajectory tube is as follows: Extract the current 3D coordinates of the global situation object Velocity vector acceleration vector The next time step is calculated by solving the state transition equation. The center skeleton coordinates of the trajectory tube : ； Introducing the error covariance matrix of a multi-sensor joint solution system Extract its largest eigenvalue In addition, by incorporating multi-factor uncertainty correction, the dynamic calculation is performed over the prediction time. Diffusion trajectory tube cross-sectional radius : ； + + ； in, The scaling factor is the preset confidence interval. For multi-factor uncertainty correction coefficients, For the category mobility uncertainty factor corresponding to the target semantic category, Deviation factor of historical behavior pattern corresponding to the target historical track. This refers to the environmental constraints and disturbance uncertainties corresponding to the current airspace environment. These are normalized weight coefficients that add up to 1.

9. The working method of the aerospace three-dimensional situation generation and display platform according to claim 4, characterized in that, The threat index is used to characterize the threat level of the target at the current moment. When performing rendering resource pre-allocation judgment, the predicted position and predicted velocity obtained based on the 3D error prediction trajectory tube are substituted into the calculation formula of the threat index to obtain the threat index. The calculation formula of the threat index is: ； in, , , These are normalized weight coefficients that add up to 1. The distance attenuation constant; This is the speed sensitivity coefficient; This is a nonlinear penalized activation function constructed for the target radial velocity.

10. The working method of the aerospace three-dimensional situation generation and display platform according to claim 4, characterized in that, The dynamic adjustment of rendering levels of detail and front-end refresh rate specifically includes: The current threat index of the global situational object determines the current rendering detail level and front-end refresh rate; The future spatial position of the target within a preset prediction time step is predicted based on the three-dimensional error prediction trajectory tube, and the spatial distance attribute corresponding to the future spatial position is substituted into the comprehensive threat index evaluation model to obtain the predicted threat index. When the predicted threat index reaches a higher threat range than the current display level, or when the current threat index shows an upward trend in multiple consecutive sampling periods and the increase is greater than the preset increase threshold, the 3D model, texture material, shader program, alarm effect or rendering cache resources corresponding to the multiple levels of detail of the predicted threat index are preloaded in advance. The preloading does not change the currently displayed rendering multi-level of detail, and the currently displayed rendering multi-level of detail is still determined by the threat range corresponding to the current threat index. when < At this time, the control 3D platform is downgraded to LOD-0 mode, loading only low-polygon geometry models, and the trajectory refresh rate is reduced. 1Hz; when < At that time, the control 3D platform is switched to LOD-1 mode, the standard texture model is loaded, and the trajectory refresh rate is set to 10Hz; when At that time, a semantic state mutation interruption occurred, the control 3D platform was upgraded to LOD-2 mode, the camera perspective was forced to track the target, a high-precision physical material model and peripheral radar alarm dome effect were loaded, and the rendering refresh rate was increased to 60Hz.

Images