Visual large model emergency management and control system for panoramic situation awareness

Abstract

The invention belongs to the technical field of artificial intelligence and computer vision, particularly relates to a visual large model emergency management and control system for panoramic situation awareness, and aims to solve the problems of weak multi-source visual data fusion, low risk deduction precision, lagging command decision and the like in urban public safety. The system comprises a global visual perception layer, a space-time alignment engine, a visual large model reasoning center, a dynamic situation map generator, a multi-granularity emergency decision module and an instruction distribution feedback unit. By constructing a high-precision space-time grid and a visual large model for joint reasoning, conversion from pixels to semantics is realized, and a time sequence evolution knowledge graph is generated and dynamically updated; and in combination with a hierarchical response mechanism and a cloud edge collaborative architecture, automatic disposal and man-machine collaborative decision making are supported, and the emergency response efficiency and the system robustness are improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence and computer vision technology, specifically relating to a large-scale visual model emergency management system for panoramic situational awareness. Background Technology

[0002] The deep integration of artificial intelligence and computer vision technologies is driving intelligent sensing systems towards higher levels of cognition and decision-making capabilities. In key areas such as public safety, urban management, and emergency command, the demand for real-time understanding and rapid response to complex and dynamic environments is increasing. Panoramic situational awareness, as a core means of achieving full-domain information fusion and macro-level decision support, has become a key direction in the construction of intelligent systems. Visual large-scale models, with their powerful representation learning capabilities, cross-scene generalization performance, and unified processing architecture for multi-source visual data, are gradually becoming the underlying technological pillar for building next-generation sensing systems. Their general visual understanding capabilities, formed through pre-training on massive amounts of data, provide a new technological path for addressing uncertainties in unstructured environments.

[0003] Among them, panoramic situational awareness systems for emergency management scenarios are required to quickly extract key event elements, identify potential risk patterns, and generate interpretable situational evolution simulation results from multi-view and multi-temporal visual data such as wide-area video surveillance, drone aerial photography, and mobile terminal feedback when emergencies occur, in order to support efficient decision-making by command personnel. Such systems need to have end-to-end identification and analysis capabilities for typical emergency events such as abnormal behavior, crowd gatherings, traffic congestion, and disaster spread, while meeting the technical requirements of low latency, high robustness, and multimodal semantic alignment, thereby achieving a leap from "seeing" to "understanding."

[0004] While existing technologies have made some progress in single-task detection or localized area monitoring, they still face multiple bottlenecks when building a unified emergency management system for panoramic situational awareness: Large visual models typically rely on large-scale static datasets for training, making them difficult to adapt to the demands of online learning amidst drastic changes in data distribution during emergencies; the high inconsistency in resolution, viewpoint, and lighting conditions among multi-source heterogeneous visual inputs leads to significant fluctuations in model inference results; insufficient modeling capabilities for complex spatiotemporally related events (such as chain reaction accidents) and a lack of effective causal inference mechanisms; high coupling in the overall system architecture, making on-demand scheduling and elastic resource scaling difficult; furthermore, in edge-cloud collaborative deployment environments, the contradiction between the computational overhead of large models and real-time requirements is prominent, severely impacting the timeliness and accuracy of emergency response. Therefore, there is an urgent need to build a large visual model-driven emergency management system with dynamic adaptability, strong robustness, and efficient inference performance to overcome the current technical bottlenecks in the practical application of panoramic situational awareness. Summary of the Invention

[0005] The purpose of this invention is to provide a large-scale visual emergency management system for panoramic situational awareness, addressing the technical challenges of weak multi-source heterogeneous visual data fusion capabilities, low accuracy in dynamic risk evolution prediction, delayed cross-level command and decision-making, and insufficient efficiency in human-machine collaborative response in current city-level public safety incident responses. Existing technologies rely on independently deployed video surveillance platforms and rule-driven alarm mechanisms, making it difficult to achieve a global understanding and proactive prediction of emergencies in complex scenarios. Furthermore, information silos exist between subsystems, failing to support the demands of high concurrency, strong real-time performance, and multimodal emergency response.

[0006] The technical solution of this invention includes a global visual perception layer, a spatiotemporal alignment engine, a large-scale visual model inference center, a dynamic situation map generator, a multi-granularity emergency decision-making module, and a command distribution and feedback closed-loop unit. The global visual perception layer consists of infrared imaging equipment, visible light high-definition cameras, drone inspection terminals, and mobile law enforcement recorders deployed at key urban nodes. It continuously collects raw visual stream data covering main traffic arteries, densely populated areas, and infrastructure areas. This visual stream data is uploaded to edge computing nodes at a frequency of no less than 25 frames per second, and preliminary denoising, compression encoding, and metadata annotation are performed locally. Furthermore, the spatiotemporal alignment engine receives visual stream data from different geographic coordinate and timestamp systems. By constructing a unified spatial reference frame and time synchronization protocol, and employing a joint optimization algorithm based on feature point matching and GNSS-assisted calibration, it maps multi-view image sequences to a three-dimensional geographic grid with centimeter-level accuracy, forming a spatiotemporally consistent input basis. As one embodiment of this invention, the GNSS-assisted calibration utilizes correction signals provided by differential positioning base stations to control the spatial registration error within 0.3 meters.

[0007] The visual large-scale model inference hub is built on the Transformer architecture, with a backbone network of no less than 1.4 billion parameters. The training corpus covers more than 20 million sets of labeled image-text pairs of urban operation scenes, supporting four core capabilities: open vocabulary object detection, behavioral semantic parsing, abnormal pattern recognition, and causal relationship inference. After receiving multi-source visual streams that have been spatiotemporally aligned, the inference hub performs an end-to-end joint inference task: first, it performs fine-grained classification and state recognition of entity objects in the image; second, it analyzes the interaction topology between objects; third, it determines whether there are behavioral combinations that violate preset safety rules; and finally, it outputs a structured event description package containing object identifiers, location trajectories, behavioral labels, confidence scores, and potential impact ranges. Furthermore, the dynamic situation map generator receives continuous event description packets output by the inference center and transforms them into temporally evolving knowledge graph nodes and edge relationships. The knowledge graph is dynamically updated using a graph neural network, performing a graph structure refresh operation every 500 milliseconds. Newly added nodes represent newly discovered abnormal entities or event clusters, and the state attributes of existing nodes are weighted and corrected based on the latest observation results. Edge weights reflect the spatial proximity, temporal order, and logical causal strength between events.

[0008] The multi-granularity emergency decision-making module calculates tiered response strategies based on the current snapshot and historical evolution path of the dynamic situation map. When a single isolated event is detected and its risk level is below threshold 3, an automated handling process is triggered, and a pre-set emergency plan template is invoked to generate operational suggestions. When multiple related events are identified as forming a complex crisis chain, and the overall risk index reaches or exceeds threshold 7, an expert collaborative consultation mode is initiated, projecting the situation map onto the command center's 3D visualization interface and pushing key simulation conclusions to the decision-makers' terminals. The risk index is a weighted composite of four dimensions: event severity, spread rate, affected population density, and resource accessibility, with the weighting coefficients dynamically adjusted according to the event type. Furthermore, the instruction distribution and feedback closed-loop unit is responsible for converting the decision results into a set of specific executable action instructions and issuing them to the corresponding execution terminals according to a priority queue mechanism. For unattended facilities, instructions directly drive camera zoom tracking, broadcast system voice prompts, or gate linkage control. For manual intervention, instructions are pushed to the nearest available emergency force terminal in the form of a structured order, along with optimal travel routes and equipment suggestions.

[0009] Preferably, the visual large-scale model inference center adopts a two-stage training paradigm: the first stage injects domain-specific corpora from the large-scale urban governance field into the general vision-language pre-trained model for domain adaptation fine-tuning; the second stage introduces adversarial example enhancement and curriculum learning strategies to focus on improving the model's robustness under adverse imaging conditions such as low light, occlusion, and long distance; during training, a focus loss function is used to alleviate the imbalance between positive and negative samples, increasing the recall rate of rare events to over 92%. Furthermore, the dynamic situation map generator incorporates a conflict resolution mechanism. When the same entity is assigned contradictory state labels within adjacent time windows, the context-aware arbitration module is activated, combining the entity's historical behavior patterns, surrounding environmental trends, and group movement consistency indicators to reassess and output the most likely true state.

[0010] Preferably, the multi-granularity emergency decision-making module integrates a reinforcement learning-driven policy optimizer, whose reward function design balances response timeliness and resource utilization. In offline environments, the policy optimizer continuously iterates and updates policy network parameters based on historical event replay data; during online deployment, it synchronizes the latest policy version every 3 minutes. When faced with new and unknown event types, the system automatically switches to an analogy-based decision-making mode based on case reasoning, retrieving successful handling solutions from the historical database under similar scenarios as initial recommendations. Furthermore, the instruction distribution and feedback closed-loop unit establishes a bidirectional communication link, requiring all execution terminals to return a confirmation status within 10 seconds of receiving an instruction. If no feedback is received, a redundant channel retry mechanism is activated, and this anomaly is included in the system reliability assessment indicators.

[0011] Preferably, the system runs on a distributed cloud-edge collaborative architecture. Edge nodes handle the preprocessing and lightweight inference of front-end visual data, while the central cloud platform is responsible for full inference of the large visual model, maintenance of the situation map, and global decision-making computation. A hierarchical transmission strategy is adopted between the edge and the cloud, uploading only high-value event fragments and summary information to the center, reducing bandwidth consumption to less than 8% of the original video stream. When the network is interrupted, the edge nodes activate local caching and autonomous decision-making modes to ensure that basic monitoring and alarm functions are not interrupted. Furthermore, the system is equipped with an adaptive load balancer that dynamically adjusts the inference frequency and resolution level of each module based on real-time computing resource usage, achieving optimal energy efficiency while ensuring the performance of critical tasks.

[0012] Compared with the prior art, the advantages and positive effects of the present invention are as follows: This solution achieves a technological leap from raw pixels to high-level semantics and then to evolutionary prediction by constructing a cascaded architecture of a large visual model inference center and a dynamic situation map generator. It breaks through the limitations of traditional video analysis systems, which are confined to static object detection and simple rule matching, enabling the system to understand and predict complex urban events. Furthermore, this solution unifies multi-source heterogeneous visual data onto a high-precision spatiotemporal grid and continuously updates the relational weights between events through graph neural networks, forming a dynamic knowledge representation system with self-evolutionary characteristics. This solves the problems of information fragmentation and missing connections in previous emergency systems, significantly improving the ability to interpret chain-reaction events. The solution improves the accuracy of disaster identification. It employs a multi-granularity decision-making mechanism, automatically selecting between fully automated response and human-machine collaborative consultation modes based on the event's risk level. Combined with a dual strategy engine of reinforcement learning and case-based reasoning, it ensures rapid closed-loop handling of routine events while providing scientific decision support for major crises, reducing the average emergency response time by over 40%. Furthermore, the solution implements a cloud-edge collaborative computing architecture and a hierarchical data transmission strategy, effectively controlling communication overhead and energy consumption while meeting high-performance inference requirements. This enhances the system's robustness and sustainable operation, making it particularly suitable for stable operation in wide-area deployments and sudden high-load scenarios. Attached Figure Description

[0013] Figure 1 This is a schematic diagram of the overall technical solution architecture proposed in this invention; Figure 2 This is a schematic diagram of the core principle framework of the cascaded architecture of the visual large model inference center and the dynamic situation map generator in this invention. Detailed Implementation

[0014] Please refer to Figure 1 and Figure 2 This invention relates to a large-scale visual model-based emergency management system for panoramic situational awareness, aiming to achieve full coverage, real-time response, and intelligent simulation of city-level public safety incidents. The system consists of a full-domain visual perception layer, a spatiotemporal alignment engine, a large-scale visual model inference center, a dynamic situation map generator, a multi-granularity emergency decision-making module, and a command distribution and feedback closed-loop unit. Each module follows a strict logical sequence for data flow and functional collaboration, forming a complete technical closed loop from raw visual signal acquisition to high-level semantic understanding, and then to cross-level command and dispatch and execution feedback.

[0015] The system's overall operation begins with data acquisition at the full-domain visual perception layer. This layer is deployed at key urban nodes, including major traffic artery intersections, entrances and exits of large commercial complexes, subway station concourses, bridge and tunnel structures, areas surrounding key cultural relics protection units, and public spaces in high-density residential areas. At these locations, four types of front-end sensing devices are deployed: infrared imaging equipment, high-definition visible light cameras, drone inspection terminals, and mobile law enforcement recorders. Each device has a unique identification number, geographic coordinate registration information, and its assigned jurisdiction, and completes self-checks, time synchronization, and network registration through an embedded operating system.

[0016] Infrared imaging equipment is primarily used for detecting heat sources in nighttime or low-light environments. Its sensor array has a resolution of 640×512 pixels, a stable frame rate of 30 frames per second, and a temperature sensitivity better than 0.08 degrees Celsius, effectively identifying the contour features of warm-blooded objects such as pedestrians and vehicles. The equipment incorporates a non-uniformity correction algorithm to automatically compensate for pixel drift caused by prolonged operation, ensuring consistent output images. The visible light high-definition camera uses 4K ultra-high-definition resolution and supports wide dynamic range imaging technology, preserving detail even in strong backlight or alternating light and dark scenes. The lens features motorized zoom and autofocus capabilities, with a field of view adjustable between 20 and 90 degrees to adapt to different monitoring distances. The camera is equipped with an audio acquisition module to simultaneously capture ambient sound information, serving as auxiliary input for subsequent behavioral analysis.

[0017] The drone inspection terminal is mounted on a multi-rotor flight platform and possesses autonomous flight path planning and obstacle avoidance capabilities. The terminal integrates a dual-light pod, containing a visible light camera and a mid-wave infrared camera, both sharing a stable gimbal to ensure consistent viewing angles. The flight platform takes off to perform aerial patrol missions according to preset cycles or triggered by unforeseen events, maintaining a flight altitude below 120 meters and a cruising speed of 15 meters per second. During missions, the drone transmits video streams in real-time to edge computing nodes via a 5G private network link, while simultaneously receiving dynamic mission adjustment instructions from the central cloud platform. The mobile law enforcement recorder is worn by frontline police officers and features image stabilization, one-button recording start, and GPS positioning. The device continuously records first-person perspective footage during duty and automatically marks key segments when it detects severe shaking or voice keywords, improving subsequent retrieval efficiency.

[0018] All raw visual stream data collected by the aforementioned devices are uploaded to the nearest edge computing node at a frequency of no less than 25 frames per second. The transmission protocol adopts a customized streaming media encapsulation format based on the User Datagram Protocol (UDP), with the packet header containing a unique device identifier, a UTC standard timestamp, geographic location coordinates, data type label, and an encryption checksum. At the edge node, the integrity of the received data packets is first verified by comparing the checksum using the SHA-256 hash algorithm. If data tampering or packet loss is detected, retransmission is requested. Subsequently, preliminary denoising processing is performed. For visible light images, a non-local mean-based filtering algorithm is applied to suppress random noise; for infrared images, a spatiotemporal joint filtering method is used to eliminate fixed-pattern noise and temporal flicker effects. The denoised images enter the compression encoding stage, using the H.265 encoding standard for efficient compression. The quantization parameters are dynamically adjusted according to the scene complexity to ensure that the clarity of key areas is maintained even under bandwidth-constrained conditions. After encoding is completed, the system automatically generates metadata labels, including: equipment type, installation azimuth angle, pitch angle, current focal length value, ambient light level, weather condition classification (sunny, cloudy, rainy, foggy), number of detected moving targets and their rough categories (people, vehicles, non-motorized vehicles).

[0019] After local preprocessing, the data is temporarily stored in the high-speed cache of the edge nodes, awaiting further processing instructions. Some lightweight analysis tasks, such as license plate recognition, face capture, and boundary crossing alarms, are completed directly by the edge nodes, and the results are reported in summary form. However, core tasks involving global understanding and complex reasoning require high-value data fragments to be uploaded to the central cloud platform for deeper fusion analysis.

[0020] Next, we move on to the second part of the system: the implementation of the spatiotemporal alignment engine. The core responsibility of this engine is to solve the inconsistency problem of multi-source heterogeneous visual data in the spatial and temporal dimensions, establish a unified spatiotemporal benchmark framework, and provide a geometric alignment basis for subsequent joint inference.

[0021] In terms of spatial dimension, the system constructs a three-dimensional geographic grid system with centimeter-level precision covering the entire city. This grid is based on a digital elevation model provided by the city's surveying department, combined with laser point cloud scanning results for refined modeling. The grid cell size is set at 0.1m × 0.1m × 0.1m, vertically covering 30 meters of space above ground, sufficient to accommodate building facades and aerial objects. Each grid cell has a unique three-dimensional coordinate index and is associated with its corresponding land feature category (roads, greenery, building walls, water bodies, etc.). When cameras from different locations capture the same physical area, the pixels in their images need to be mapped to their corresponding spatial locations within this unified grid.

[0022] To address this, the system employs a joint optimization algorithm based on feature point matching and GNSS-assisted calibration. First, computer vision technology is used to extract stable feature points within the field of view of each camera, such as building corners, lamppost tops, and traffic sign edges—landmarks with significant geometric features and low occlusion. Feature descriptors are extracted using the SIFT algorithm, and a city-wide landmark feature database is established on the server. Whenever a new device is connected or an existing device is displaced, the system automatically initiates a calibration process: continuously acquiring video sequences over a period of time, extracting feature points, comparing them with the database, and calculating the extrinsic parameter matrix (rotation and translation parameters) of the device relative to the 3D mesh. This process is repeated multiple times to improve estimation accuracy, ultimately yielding sub-pixel level registration results.

[0023] To further improve spatial registration accuracy, the system introduces a GNSS-assisted calibration mechanism. Each fixed camera and UAV terminal integrates a high-precision GNSS receiver module, supporting dual-mode positioning with BeiDou-3 and GPS. Differential positioning base stations are deployed at high points in the city, broadcasting pseudorange correction signals every second to reduce terminal positioning errors to within 0.1 meters. When there is a deviation between the device's own GNSS positioning coordinates and its spatial position calculated from image features, the system activates a joint optimization solver to construct a least-squares objective function, simultaneously optimizing the pose parameters of all relevant devices to minimize the overall projection error. As one embodiment of the invention, the GNSS-assisted calibration utilizes the correction signals provided by the differential positioning base stations to control the spatial registration error within 0.3 meters.

[0024] In the time dimension, the system establishes a strict time synchronization protocol. All front-end devices and edge nodes are connected to the network time protocol server, using the PTP precision time protocol for time synchronization, achieving a clock synchronization accuracy of ±1 microsecond. Each frame of image is precisely timestamped before encoding, in UTC millisecond format. When multi-view image sequences flow into the spatiotemporal alignment engine, the system first sorts them by timestamp, then aligns the data streams with a minimum time window of 1 millisecond. For minor frame errors caused by network latency, linear interpolation is used to fill in the missing time point state estimates. After the above processing, all visual stream data is uniformly mapped to a two-dimensional "space-time" coordinate system, forming a spatiotemporally consistent input basis, providing a reliable guarantee for subsequent multi-view fusion inference.

[0025] Building upon this foundation, the system enters its third core component: the visual large-scale model inference hub. This hub is constructed based on the Transformer architecture, with a backbone network of 1.4 billion parameters, employing a 24-layer depth and 16 attention heads, with each feedforward network layer having a width of 4096. The model training corpus exceeds 20 million pairs of images and text pairs, covering various typical scenarios such as urban traffic congestion, crowd gatherings, violent conflicts, fire smoke, illegal road occupation, facility damage, and abnormal parking. The text descriptions are written by a professional annotation team, adhering to unified grammatical standards and including elements such as objects, actions, scenes, and causal relationships.

[0026] After receiving the spatiotemporally aligned multi-source visual stream, the reasoning center executes an end-to-end joint reasoning task. The entire reasoning process consists of four progressive stages: The first stage is fine-grained entity recognition. The model segments and classifies each frame of the image, identifying all visible object instances in the scene, including but not limited to pedestrians, motor vehicles, non-motor vehicles, animals, obstacles, fire hydrants, manhole covers, billboards, etc. The classification system adopts an open vocabulary design, supporting zero-shot transfer capability, that is, for new categories not appearing in the training set (such as new types of transportation), it can also make reasonable inferences through semantic similarity matching. Each recognition result is accompanied by bounding box coordinates, category label, confidence score (ranging from 0 to 1), and state attributes (such as "stationary", "walking", "running", "falling down", "climbing over", etc.).

[0027] The second stage is interaction topology analysis. The model further analyzes the spatial relationships and behavioral interaction patterns between objects. For example, it determines whether two pedestrians have physical contact, whether a vehicle is closely following another vehicle, or whether a drone is approaching a sensitive area. The system constructs a local graph structure, where nodes represent identified objects and edges represent their relative positions, movement trends, and potential interaction strengths. A graph attention mechanism is used to aggregate neighborhood information, enhancing the understanding of complex group phenomena.

[0028] The third stage is the detection of safety rule violations. The system has a built-in, scalable safety rule knowledge base, containing formalized expressions of legal provisions such as the "Regulations on the Safety Management of Public Places," the "Road Traffic Safety Law," and the "Measures for the Safety Management of Large-Scale Public Events." Each rule defines prohibited combinations of behaviors and their applicable conditions. For example, "staying within the yellow line of a subway platform for more than 30 seconds without any companions nearby" is considered a high-risk loitering behavior; "three or more motorcycles driving side by side at a speed exceeding 60 kilometers per hour" constitutes suspected illegal street racing. The model performs pattern matching between the currently observed behavioral label sequences and the rule base, and once a match is found, it is marked as a candidate for an abnormal event.

[0029] The fourth stage is causal inference. The model attempts to establish logical chains between events and predict the subsequent developments that current behavior may lead to. For example, if it detects someone throwing an object at a crowd, it immediately activates a risk warning for a potential stampede; if it observes cracks in a building's exterior wall accompanied by signs of rainwater leakage, it infers a potential risk of wall collapse. This stage relies on prior knowledge of common sense and physical laws learned during the pre-training stage, combined with contextual information, to perform inductive reasoning.

[0030] Finally, the inference center outputs a structured event description package, whose data structure is defined as follows: It includes a list of object identifiers, an array of their respective location trajectories (sampling interval of 200 milliseconds), a set of behavioral labels, a confidence score vector, a potential impact area polygon (represented by geographic coordinates), an event type classification (single event / compound event), an urgency level (levels 1 to 10), and a suggested attention priority (high / medium / low). This description package is encapsulated in JSON format and pushed to the next processing unit via a message queue middleware.

[0031] To further enhance the model's robustness in real-world environments, the visual large-scale model's inference center employs a two-stage training paradigm. The first stage is domain-adaptive fine-tuning, where a large-scale corpus specific to the urban governance domain is injected into the general vision-language pre-trained model for retraining. This corpus comprises 8 million sets of data, sourced from publicly available government datasets, anonymized versions of historical police reports, and simulated drill videos, covering special scenarios such as extreme weather, major holidays, and emergencies. During fine-tuning, the weights of the underlying convolutional layers are frozen, and only the top-level attention module and classification head parameters are updated to avoid catastrophic forgetting.

[0032] The second phase is adversarial enhancement training, focusing on improving the model's performance under harsh imaging conditions such as low light, occlusion, and long distance. The system generates various types of adversarial examples, including adding Gaussian noise, simulating fog attenuation, partially occluding targets, reducing resolution to below 720p, and inserting false interference. A course-based learning strategy is adopted, gradually increasing the difficulty of the examples from easy to hard, allowing the model to gradually adapt to complex environments. During training, a focus loss function is used to alleviate the imbalance between positive and negative samples, especially increasing gradient weights for rare events (such as armed fights and arson), making the model more attentive to low-frequency but high-risk situations. Experimental data shows that after optimization using this training paradigm, the model's recall rate for rare events is improved to over 92%, while the false positive rate is controlled below 5%.

[0033] Furthermore, the system integrates a context-aware arbitration module within its inference center to handle situations where the same entity is assigned contradictory state labels within adjacent time windows. For example, someone might be identified as "walking normally" in one frame but misjudged as "falling down" in the next. In this case, the system activates a conflict resolution mechanism, comprehensively considering three indicators: the entity's historical behavior patterns (whether it has frequently fallen in the past 5 minutes), the trend of changes in the surrounding environment (whether others have approached or pushed), and the consistency of group movement (whether there has been panicked escape among the surrounding crowd). A weighted voting mechanism is used to reassess the most likely true state and output a revised label, ensuring the consistency and credibility of the event description.

[0034] Next is the fourth functional module: the dynamic situation graph generator. This generator receives continuous event description packets output by the inference center and transforms them into a temporally evolving knowledge graph structure. The graph is represented using an attribute graph model, where nodes represent anomalous entities or event clusters, and edges represent the relationships between them.

[0035] Each newly created node is assigned a set of initial attributes upon creation: node ID, entity type, first discovery time, geographic center point, initial confidence level (derived from the inference center output), links to related evidence fragments, and handling status (no response / dispatched / processing / closed). The status attributes of existing nodes are dynamically updated based on the latest observations. The system uses an exponentially weighted moving average algorithm to smooth the confidence level, as shown in the following formula: in, The confidence level at the current moment. The confidence level is based on the latest observations. The historical confidence level at the previous moment. The learning rate is 0.6, which means that the system gives high weight to new evidence but does not overly rely on a single observation.

[0036] The edge relationships in the graph reflect three core associations: spatial proximity, temporal sequence, and logical causal strength. Spatial proximity is determined by calculating the Euclidean distance between the geographic centers of two nodes; if the distance is less than 50 meters, a connection is established, and the edge weight is inversely proportional to the distance. Temporal sequence is determined by the time difference between two events; if the latter occurs within 10 minutes of the former, it is considered to have potential temporal continuity, and the edge weight decreases as the time interval increases. Logical causal strength is evaluated by the semantic reasoning engine. For example, "traffic accident" and "traffic congestion" have a strong causal relationship, with a weight of 0.9; while "square dancing activities" and "streetlight malfunction" have no obvious connection, and their weights approach 0.

[0037] The graph neural network is responsible for dynamically updating the graph structure. The system performs a graph refresh operation every 500 milliseconds, executing the following steps: scanning all newly arrived event description packets and creating candidate nodes for each newly discovered anomaly; traversing the existing node set to find potential related targets; calculating the connection probability based on the three types of association rules mentioned above, and establishing new edges if the comprehensive score exceeds a threshold of 0.5; decaying the weights of existing edges with a decay coefficient of 0.1 per minute to simulate the natural fading of event influence; and finally, launching a community detection algorithm to identify highly interconnected subgraph structures and cluster them into "event clusters," which serve as higher-level cognitive units for the decision-making module.

[0038] When multiple related events are detected to form a complex crisis chain, the system automatically constructs an evolutionary path prediction model. Starting from the current snapshot of the event map, and combining historical evolution patterns with external environmental variables (such as weather forecasts and holiday schedules), a time series prediction method based on a gated graph neural network is used to extrapolate possible development trends within the next 15 minutes. The prediction results are presented in the form of probability distributions, such as "the probability of the fire spreading to adjacent buildings is 78%" and "the risk level of stampede will rise to level 9 in 8 minutes," providing a scientific basis for proactive intervention.

[0039] The fifth module is the multi-granularity emergency decision-making module. This module calculates tiered response strategies based on the current snapshot and historical evolution path of the dynamic situation map. The system defines risk indices. As a core indicator triggering decision-making, its calculation method is a weighted composite of four dimensions: in, Indicates the severity of the event, with a value from 1 to 10, determined based on the event type and the scope of its direct impact; Indicates the spread rate, measured in meters per minute, reflecting the rate at which a threat propagates. This indicates the affected population density, expressed in people per square meter, derived from urban population heat map data. Indicates resource accessibility, with a value between 0 and 1, representing the reciprocal normalized value of the time required for the nearest emergency response force to arrive at the scene; weighting coefficient. to The adjustment is dynamically based on the type of event; for example, fire-related events will be emphasized. and Mass incidents are more concerned with .

[0040] When a single isolated event is detected and its risk index is below the threshold of 3, the system classifies it as a low-risk disturbance and triggers an automated response process. At this point, the system calls upon the corresponding solution from a pre-set emergency response plan template library to generate operational suggestions. The plan templates are stored in XML format and include fields such as trigger conditions, response action sequence, responsible party, required resource list, and expected effect assessment. For example, for a "missing manhole cover" event, the system automatically generates the following instructions: "Activate flashing road warning lights—Issue a municipal maintenance work order—Push detour instructions to the navigation app—Close nearby drainage outlets to prevent falls."

[0041] When multiple related events are identified as forming a complex crisis chain, and the overall risk index reaches or exceeds the threshold of 7, the system switches to expert collaborative consultation mode. At this time, a dynamic situation map is projected onto the command center's 3D visualization interface, displaying the spatial distribution, evolution path, and resource deployment of events in a virtual reality format. Simultaneously, key simulation conclusions are pushed to decision-makers' terminals, including: predictions of the highest risk area, optimal evacuation route suggestions, priority ranking of emergency force deployment, and early warnings of potential secondary disasters. Decision-makers can interactively adjust these conclusions on the interface, and the system provides real-time feedback on the modified simulation results, assisting in reaching an optimal consensus on decision-making.

[0042] To further improve decision-making quality, the multi-granularity emergency decision-making module integrates a reinforcement learning-driven policy optimizer. This optimizer continuously iteratively updates the policy network parameters in an offline environment based on historical event replay data. Each historical case includes a complete event sequence, the actions taken, the final outcome evaluation (success / failure / partial success), and a social impact assessment. The reward function is designed to balance response timeliness and resource utilization, defined as: in, For instant reward value, The time required for the event to close the loop. Countdown to response time To ensure full utilization of resources, For the cost of incorrect response, , , To adjust the coefficients, the policy network employs a deep Q-network structure, exploring the action space through Monte Carlo tree search to find the decision path with the highest long-term reward. During online deployment, the latest policy version is synchronized from the central server every 3 minutes to ensure real-time absorption of the latest experience.

[0043] When faced with new and unknown event types, the system automatically switches to an analogical decision-making mode based on case-based reasoning. It retrieves past cases similar to the current situation from the historical database, calculating similarity based on the cosine distance of event feature vectors. Features include location type, time rhythm, composition of involved parties, and initial behavioral patterns. The top five most similar cases are selected, and their successful handling solutions are extracted as initial recommendations. These are then adaptively adjusted based on the current environmental differences to form a preliminary response strategy.

[0044] The sixth module is the instruction distribution and feedback closed-loop unit. This unit is responsible for converting the decision results into a set of specific executable action instructions and issuing them to the corresponding execution terminals according to a priority queue mechanism. The instruction set is written in a standardized command language and includes fields such as operation type, target device ID, execution parameters, deadline, and security check code.

[0045] For unattended facilities, commands directly drive equipment responses. For example, a "zoom tracking" command can be sent to a specific camera to automatically lock onto high-risk individuals and keep the image centered; a "voice prompt" command can be sent to a public address system to play pre-recorded audio to guide crowd evacuation; and a "linkage control" command can be sent to a smart gate to open emergency exits or restrict access. All actions are protected by hardware-level interlocks to prevent secondary risks caused by misoperation.

[0046] For manual intervention, instructions are pushed to the nearest available emergency response terminal in the form of structured work orders. The system calculates the shortest path between the current location of each duty unit and the incident site based on a GIS geographic information system, and estimates arrival time by considering road conditions and vehicle type (walking, electric vehicles, patrol cars). Priority is given to assigning the personnel with the shortest estimated arrival time and the appropriate qualifications to handle the situation. The work order includes a summary of the incident details, screenshots of on-site images, optimal route navigation, a recommended equipment list (such as riot control forks, first aid kits, and fire extinguishers), and communication channel settings suggestions.

[0047] This unit establishes a two-way communication link, requiring all execution terminals to return an acknowledgment status within 10 seconds of receiving the instruction. The acknowledgment information includes the reception timestamp, the executor's authentication code, current location, and estimated arrival time. If no feedback is received within the specified time, the system activates a redundant channel retry mechanism, retransmitting the instruction sequentially via the 5G private network, LTE backup link, and satellite communication, while increasing the task's priority. After three consecutive failures, the system incorporates this anomaly into its reliability assessment metrics for subsequent optimization of resource allocation and communication strategies.

[0048] The system operates on a distributed cloud-edge collaborative architecture. Edge nodes are deployed in district-level data centers, undertaking preprocessing and lightweight inference tasks for front-end visual data, including motion detection, target capture, and simple alarm generation. The central cloud platform is located in the city-level main data center, equipped with a high-performance GPU cluster, responsible for full inference of the large visual model, maintenance of dynamic situation maps, and global decision-making calculations. A tiered transmission strategy is adopted between the edge and the cloud: under normal conditions, only high-value event segments (such as 30 seconds of video before and after the detection of abnormal behavior) and structured summary information are uploaded; for normal monitoring streams, only keyframe thumbnails and metadata indexes are retained, reducing bandwidth usage to less than 8% of the original video stream.

[0049] When encountering network outages or congestion, edge nodes activate local caching and autonomous decision-making modes. Local storage capacity is no less than 64TB, capable of storing complete video recordings for the most recent 72 hours. A lightweight decision-making model is also loaded, supporting rule-based local alarms and device linkage to ensure uninterrupted basic monitoring and alarm functions. Once the network is restored, cached data and event logs are automatically synchronized to the cloud to complete historical records.

[0050] The system is equipped with an adaptive load balancer that monitors the CPU utilization, memory usage, GPU memory consumption, and network throughput of each module in real time. When computing resources are strained, the inference frequency and resolution level of each module are dynamically adjusted. For example, during peak periods, the image analysis frame rate for non-critical areas is appropriately reduced to 15 frames per second, or the search range for object detection is narrowed to prioritize the processing performance of core areas and high-risk events. In energy-constrained scenarios, the model inference precision mode can also be reduced (switching from FP32 to FP16) to achieve optimal energy efficiency.

[0051] Through the close collaboration of the six modules mentioned above, this system achieves a technical closed loop from raw visual signals to high-level cognitive understanding and precise command and dispatch. Compared to the limitations of traditional video surveillance systems, which are limited to static target detection and simple rule matching, this solution leverages the powerful semantic parsing capabilities of large-scale visual models to overcome the bottleneck in understanding complex events. The continuous evolution mechanism of the dynamic situation map solves the problems of fragmented information and missing connections in previous emergency systems, significantly improving the accuracy of identifying chain-reaction disasters. The multi-granularity decision-making mechanism automatically selects the response mode based on risk level, ensuring rapid closed-loop handling of routine events while providing scientific auxiliary decision support for major crises. The cloud-edge collaborative architecture and hierarchical transmission strategy effectively control communication overhead and energy consumption while meeting high-performance requirements, enhancing the system's robustness and sustainable operation. Actual deployment tests show that this system reduces the average emergency response time by 43%, decreases the misjudgment rate of major events to 2.1%, and improves resource scheduling efficiency by 37%, making it particularly suitable for stable operation in wide-area deployments and sudden high-load scenarios.

Claims

1. A visual large-scale model emergency management system for panoramic situational awareness, characterized in that, include: The global visual perception layer is used to collect multi-source raw visual stream data covering multiple key areas; The spatiotemporal alignment engine is used to receive the multi-source raw visual stream data and, by constructing a unified spatial reference frame and time synchronization protocol, map the visual stream data from different perspectives and timestamps to a unified three-dimensional geographic grid, forming spatiotemporally consistent input data. The visual large model inference center is used to receive the spatiotemporally consistent input data, perform end-to-end joint inference tasks, perform fine-grained recognition and state determination of entity objects in the picture, analyze the interaction topology between objects, detect the combination of behaviors that violate preset security rules, and output a structured event description package containing object identifiers, location trajectories, behavior tags and potential impact ranges. A dynamic situational graph generator is used to receive the structured event description package, transform it into a temporally evolving knowledge graph, wherein the nodes of the knowledge graph represent abnormal entities or event clusters, the edge relationships reflect the spatial proximity, temporal order and logical causal strength between events, and a graph neural network is used to dynamically update the knowledge graph. The multi-granularity emergency decision-making module is used to calculate the event risk index based on the current snapshot and historical evolution path of the knowledge graph, and implement a graded response strategy according to the level of the risk index to generate decision results. The instruction distribution and feedback closed-loop unit is used to convert the decision result into an executable set of action instructions and issue them to the corresponding unattended facilities or manual emergency force terminals according to priority, while establishing a two-way communication link to receive execution feedback.

2. The visual large-scale model emergency management system for panoramic situational awareness as described in claim 1, characterized in that, The global visual perception layer includes: infrared imaging equipment deployed at key urban nodes, visible light high-definition cameras, drone inspection terminals, and mobile law enforcement recorders.

3. The visual large-scale model emergency management system for panoramic situational awareness as described in claim 1, characterized in that, The spatiotemporal alignment engine includes: a feature point matching unit for extracting and matching stable landmark feature points in each visual stream; a GNSS-assisted calibration unit for receiving differential positioning correction signals and calibrating the device pose; and a joint optimization unit for combining feature point matching results with GNSS calibration data to solve for the optimal spatial extrinsic parameter matrix and achieve high-precision registration.

4. The visual large-scale model emergency management system for panoramic situational awareness as described in claim 1, characterized in that, The visual big data model reasoning center includes: an open vocabulary target detection unit, used to identify entity objects in the image and assign category labels; a behavior semantic parsing unit, used to determine the state attributes of the entity objects and their interaction patterns; an abnormal pattern recognition unit, used to match the observed behavior patterns with a preset security rule knowledge base to identify abnormal events; and a causal relationship inference unit, used to infer the logical causal chain between events based on common sense and contextual information.

5. The visual large-scale model emergency management system for panoramic situational awareness according to claim 4, characterized in that, The visual big model reasoning center also includes a context-aware arbitration module, which is used to re-evaluate and output the most likely true state when the same entity is assigned contradictory state labels in adjacent time windows, by combining the entity's historical behavior patterns, the trend of changes in the surrounding environment, and the consistency index of group movement.

6. The visual large-scale model emergency management system for panoramic situational awareness as described in claim 1, characterized in that, The dynamic situation map generator includes: a node creation unit for creating map nodes for newly discovered abnormal entities or event clusters; a node state update unit for weighted correction of the state attributes of existing nodes based on the latest structured event description package; and an edge relationship construction unit for calculating and establishing connection edges and their weights between nodes based on the spatial distance, time interval, and semantic causal strength between events.

7. The visual large-scale model emergency management system for panoramic situational awareness as described in claim 1, characterized in that, The multi-granularity emergency decision-making module includes: a risk index calculation unit, used to synthesize an event risk index based on four dimensions: event severity, spread rate, affected population density, and resource accessibility; an automated handling unit, used to generate operational suggestions by calling a pre-set emergency plan template when the risk index is below a first threshold; and an expert collaborative consultation unit, used to project a dynamic situation map onto a three-dimensional visualization interface and push key deduction conclusions to the decision-maker's terminal when the risk index reaches or exceeds a second threshold.

8. The visual large-scale model emergency management system for panoramic situational awareness according to claim 7, characterized in that, The multi-granularity emergency decision-making module further includes: a reinforcement learning strategy optimizer, used to iteratively update the decision-making strategy based on historical event replay data in an offline environment; and a case reasoning decision-making unit, used to retrieve successful handling solutions in similar situations from the historical database as initial recommendations when facing new unknown event types.

9. The visual large-scale model emergency management system for panoramic situational awareness as described in claim 1, characterized in that, The instruction distribution and feedback closed-loop unit includes: an instruction generation unit, used to convert decision results into a standardized set of action instructions; a priority queue scheduling unit, used to sort instructions according to their urgency; and an execution end communication unit, used to directly send equipment control instructions to unattended facilities, or push structured work orders containing travel routes and equipment suggestions to manual emergency response terminals, and receive confirmation status returned by the execution end.

10. The visual large-scale model emergency management system for panoramic situational awareness according to claim 1, characterized in that, The system runs on a distributed cloud-edge collaborative architecture, where edge nodes are responsible for the preprocessing and lightweight inference of front-end visual data, and adopt a hierarchical transmission strategy to upload only high-value event fragments and summary information to the central cloud platform; the central cloud platform is responsible for the full inference of the large visual model, the maintenance of dynamic situation maps, and global decision calculation; the system is also equipped with an adaptive load balancer, which dynamically adjusts the inference frequency and resolution level of each module according to the real-time computing resource usage.

Images