A multi-modal event primitive extraction, encoding and collaborative generation method and system for smart engineering construction scenarios

Abstract

The application discloses a kind of multi-modal event primitive extraction, coding and collaborative generation method and system for smart engineering construction scene, the system includes multi-modal data acquisition module, edge preprocessing module, event candidate extraction module, time-space alignment module, subject association module, event primitive coding module, multi-source fusion generation module and primitive output module. Through edge pre-processing, candidate event extraction, time alignment, space mapping, subject association, unified coding and multi-source fusion to video, environmental parameters, device status, positioning information, wearable terminal data and job log data and other multi-modal raw data, generate standardized event primitives, and output to risk identification, knowledge graph and early warning module. The present application can reduce the transmission pressure of raw data, improve the unified semantic expression ability of multi-source heterogeneous data, the fusion accuracy and the subsequent risk analysis efficiency, suitable for hoisting, high altitude, closed space and warehousing logistics and other smart engineering construction scenes.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, edge computing, multimodal information processing, engineering safety management and industrial internet, and in particular to a method and system for extracting, encoding and collaboratively generating multimodal event primitives for smart engineering construction scenarios. Background Technology

[0002] In smart engineering construction scenarios, there are high-frequency dynamic interactions among construction workers, machinery and equipment, work areas, environmental parameters, and construction procedures. On-site, multiple heterogeneous data sources are typically deployed simultaneously, including video acquisition equipment, environmental sensors, equipment status acquisition terminals, positioning devices, wearable terminals, and work log systems. These data sources differ significantly in sampling frequency, data structure, semantic granularity, and spatiotemporal coordinate systems. Directly uploading all raw data to a central processing center for unified processing can easily lead to high bandwidth consumption, slow response times at edge nodes, and high costs for subsequent risk identification calculations.

[0003] Existing technologies typically handle multi-source heterogeneous data in engineering scenarios in the following ways: First, directly performing target detection and behavior recognition on the video stream and outputting a single alarm result; second, analyzing environmental sensor data, device logs, and positioning data as independent channels, and then performing simple stitching at a higher layer; third, compressing the raw data at the edge and uploading it to the cloud for subsequent inference. While these methods can achieve anomaly detection or risk identification to some extent, they generally lack a unified event representation method suitable for smart engineering scenarios, making it difficult to effectively align, fuse, and reuse data from different modalities.

[0004] Specifically, existing technologies have at least the following shortcomings: First, the raw data volume is large and the structure is inconsistent, which is not conducive to achieving lightweight real-time processing on the edge or terminal side; Second, the output results from different data sources are often inconsistent in granularity, with video outputting target boxes and behavior labels, sensors outputting numerical sequences, device logs outputting status codes, and positioning modules outputting coordinate points, making it difficult to form a unified computable semantic object; Third, there is a lack of structured expressions for the multi-entity interaction relationships of people, machines, environment, and tasks in engineering scenarios, making it difficult to directly utilize subsequent rule reasoning, knowledge graph modeling, and risk propagation analysis; Fourth, there is a lack of time alignment, spatial mapping, subject association, and conflict resolution mechanisms for multimodal candidate events, resulting in poor stability and insufficient interpretability of the fusion results; Fifth, most existing solutions only output terminal results of "whether an alarm is triggered," failing to provide a unified intermediate semantic carrier between the upstream identification module and the downstream early warning module.

[0005] Therefore, it is necessary to propose a method and system for extracting, encoding, and collaboratively generating multimodal event primitives for smart engineering construction scenarios, so as to convert multi-source heterogeneous raw data into structured event primitives that can be uniformly processed, transmitted across modules, and support graph and rule reasoning, thereby improving the real-time performance, accuracy, and interpretability of subsequent risk identification, early warning, and intervention systems. Summary of the Invention

[0006] Technical problems to be solved: In view of the problems existing in the existing technology, such as large amount of multimodal raw data, heterogeneous structure, difficulty in spatiotemporal alignment, insufficient semantic abstraction, and difficulty in directly supporting subsequent risk identification, knowledge graph modeling and early warning reasoning, this invention provides a method and system for multimodal event primitive extraction, encoding and collaborative generation for smart engineering construction scenarios, so as to realize unified event-based expression of multi-source heterogeneous data such as video, audio, environmental parameters, equipment status, positioning information, wearable terminal data and operation log data, and improve the real-time performance, structure, integrability and computability of event primitive generation.

[0007] Technical Solution: To solve the above-mentioned technical problems, the present invention provides a multimodal event primitive extraction, encoding and collaborative generation system for smart engineering construction scenarios, including a multimodal data acquisition module, an edge preprocessing module, an event candidate extraction module, a spatiotemporal alignment module, a subject association module, an event primitive encoding module, a multi-source fusion generation module and a primitive output module.

[0008] The multimodal data acquisition module is used to collect video data, image data, audio data, environmental sensor data, equipment status data, positioning data, wearable terminal data, and / or work log data in the smart engineering construction scenario, and send the data to the edge preprocessing module according to a predetermined format. Preferably, the environmental sensor data includes one or more of temperature, humidity, dust concentration, harmful gas concentration, noise level, vibration level, and illuminance level; the equipment status data includes one or more of equipment start / stop status, operating mode, speed, current, voltage, load rate, and fault codes.

[0009] The edge preprocessing module is used to perform lightweight processing on multimodal data. This lightweight processing includes at least one or more of the following: timestamp normalization, data cleaning, noise suppression, outlier correction, target object extraction, behavior state extraction, device state standardization, and preliminary spatial coordinate mapping. Preferably, the edge preprocessing module is deployed on an edge server, industrial gateway, on-site computing power box, or intelligent terminal with inference capabilities to reduce the bandwidth burden caused by uploading the entire original data.

[0010] The event candidate extraction module is used to generate event candidate segments based on the multimodal features after edge preprocessing. Preferably, the event candidate extraction module works in conjunction with a preset triggering rule and / or a recognition model: the preset triggering rule is used to trigger candidate events from device logs, environmental threshold changes, location boundary violations, and work process state transitions; the recognition model is used to identify personnel behavior, equipment actions, and environmental anomalies from video frame sequences, audio clips, and wearable terminal behavior data, thereby generating event candidate segments. The event candidate segment includes at least the event candidate type, candidate subject, candidate object, original time interval, original spatial location, and candidate confidence level.

[0011] The spatiotemporal alignment module is used to perform time alignment and spatial mapping on event candidate fragments generated by different modalities. Preferably, the time alignment includes time scale correction based on a unified system clock, candidate fragment aggregation based on a sliding time window, and asynchronous data correction based on delay compensation parameters; the spatial mapping includes mapping the local coordinates generated by different acquisition devices to a unified coordinate system of the engineering scene, and performing spatial positioning based on area boundaries, polygon fences, device location reference points, and personnel trajectories.

[0012] The subject association module is used to map event candidate fragments from different modalities to a unified subject entity or set of entities. Preferably, the subject association module performs subject matching based on personnel identification, equipment identification, trajectory continuity, spatial adjacency, task number, and temporal overlap to identify whether events described by different data sources refer to the same person, the same equipment, the same area event, or the same complex operation process.

[0013] The event primitive encoding module is used to encode aligned and associated event candidate fragments according to a unified event primitive structure to generate standardized event primitives. Preferably, the event primitive structure includes at least eight of the following: primitive number, event type field, event subject field, event object field, start time field, end time field, spatiotemporal location field, confidence level field, initial risk value field, data source field, context field, and rule label field. Preferably, the context field is used to record the current construction stage, regional operating conditions, environmental status summary, and process label; the rule label field is used to record the safety rules, operating specifications, or interlocking constraint identifiers associated with the event primitive.

[0014] The multi-source fusion generation module is used to perform redundancy elimination, conflict resolution, confidence correction, and fusion output on event primitives generated from different modalities. Preferably, the multi-source fusion generation module performs event primitive fusion based on event type consistency, subject consistency, spatiotemporal overlap, context similarity, and data source confidence; when multiple primitives describe the same event, redundancy merging is performed; when multiple primitives conflict in event type, spatiotemporal location, or subject identifier, conflict resolution is performed based on rule priority, confidence threshold, data source level, and historical verification results.

[0015] The primitive output module is used to output the fused target event primitives to the risk identification module, knowledge graph module, rule reasoning module, early warning module, auditing module, and / or cloud storage module. Preferably, the primitive output module supports uploading event primitives after compression and encoding according to scenario templates to further reduce bandwidth consumption.

[0016] This invention also provides a method for extracting, encoding, and collaboratively generating multimodal event primitives for smart engineering construction scenarios, comprising the following steps: S101: Collect multimodal raw data in the smart engineering construction scenario, wherein the multimodal raw data includes at least one or more of the following: video, environmental parameters, equipment status, positioning information, wearable terminal data, and work logs; S102: Perform edge preprocessing on the original multimodal data to obtain standardized multimodal feature data; S103: Extract event candidate fragments from the multimodal feature data according to preset triggering rules and / or recognition models; S104: Perform time alignment, spatial mapping, and subject association on the candidate event fragments to obtain the event object to be encoded; S105: Encode the event object to be encoded according to the unified event primitive structure to generate standardized event primitives; S106: Redundancy elimination, conflict resolution, and fusion are performed on event primitives generated in different modalities to obtain the target event primitive; S107: Output the target event primitive to the upper-level risk identification, knowledge graph modeling, rule reasoning and early warning intervention module.

[0017] Preferably, in step S103, the extraction of event candidate segments is performed using a dual-channel approach of "threshold triggering + model judgment". For environmental parameters and device status data, candidate events are generated primarily based on threshold exceeding limits, state transitions, interlocking anomalies, and duration conditions; for video, audio, and wearable terminal data, candidate events are generated primarily based on behavior recognition, target tracking, posture detection, abnormal action judgment, and voice anomaly recognition results; then, the dual-channel candidate results are uniformly input into step S104.

[0018] Preferably, in step S104, time alignment uses a unified time window mechanism to aggregate candidate events of different modalities, spatial mapping uses a scene registration matrix to map local acquisition coordinates to a unified engineering scene coordinate system, and subject association is matched based on entity identifier, trajectory continuity, and operation relationship diagram, thereby reducing the mismatch problem of candidate events from different devices or different acquisition times.

[0019] Preferably, in step S105, the event primitive adopts an expandable field structure, and the activation of fields is adaptively determined according to the scenario template. For high-altitude operation scenarios, the safety rope status field, operation height field, and restricted area field can be activated; for hoisting operation scenarios, the boom status field, hoisting radius field, and hazardous area overlap field can be activated; for confined space operation scenarios, the hazardous gas field, ventilation status field, and work permit field can be activated.

[0020] Preferably, in step S106, for a composite risk event described by multiple data sources, a composite event primitive is generated by fusing event primitives; the composite event primitive not only retains the important attributes of each original source event, but also adds a composite relationship label and a fusion confidence field, thereby providing a unified input for subsequent risk propagation analysis and graded early warning.

[0021] Preferably, the multi-source fusion generation module also supports low-confidence event compensation. When the confidence of a certain single-modal event primitive is lower than the first threshold but has significant spatiotemporal consistency and subject consistency with other modal event primitives, the system improves the fusion confidence of the event primitive through weighted compensation. When there are significant conflicts between multiple event primitives and they cannot be resolved according to existing rules, the system outputs the conflicting primitives as objects to be reviewed for manual confirmation or further judgment by the upper-level model.

[0022] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the above-described method.

[0023] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0024] Beneficial effects: Compared with the prior art, the present invention has at least the following beneficial effects: 1. By uniformly converting multimodal raw data into standardized event primitives, the problems of inconsistent granularity, inconsistent structure, and difficulty in direct reuse of results from different modalities are solved, providing a unified semantic input for subsequent risk identification, knowledge graph modeling, and rule reasoning; 2. By performing event candidate extraction and primitive encoding at the edge, the full upload of large amounts of raw data is avoided, significantly reducing bandwidth consumption and processing pressure on the central side, and improving real-time response capabilities in smart engineering construction scenarios; 3. By using time alignment, spatial mapping, and subject association mechanisms, multi-source heterogeneous data such as video, device logs, environmental sensing, positioning, and wearable terminals are effectively coupled to improve the accuracy and stability of event primitive generation; 4. By designing the event primitive field structure, the generated results simultaneously possess attributes such as event type, subject object, spatiotemporal location, confidence level, initial risk value, and rule label, thereby enhancing the structured nature, interpretability, and computability of event expression; 5. Through multi-source fusion, redundancy elimination, and conflict resolution mechanisms, single-source event primitives and compound event primitives can be output, improving the ability to express compound risks in complex construction scenarios and the accuracy of subsequent early warnings; 6. Through scene template adaptation and low confidence compensation mechanism, this invention can be applied to various smart engineering scenarios such as hoisting operations, high-altitude operations, confined space operations, and warehousing and logistics operations, and has good scalability and engineering implementation capabilities. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall system architecture according to an embodiment of the present invention; Figure 2 This is a schematic diagram of a multimodal event primitive extraction, encoding, and collaborative generation method according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the event primitive structure according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the multi-source event primitive fusion output according to an embodiment of the present invention.

[0026] exist Figure 1 In the diagram, 101 represents the multimodal data acquisition module, 102 represents the edge preprocessing module, 103 represents the event candidate extraction module, 104 represents the spatiotemporal alignment and subject association module, 105 represents the event primitive encoding module, 106 represents the multi-source fusion generation module, and 107 represents the primitive output module.

[0027] exist Figure 2 In the diagram, S101 represents the multimodal data acquisition step, S102 represents the edge preprocessing step, S103 represents the event candidate extraction step, S104 represents the spatiotemporal alignment and subject association step, S105 represents the event primitive encoding step, S106 represents the multi-source fusion generation step, and S107 represents the primitive output step.

[0028] exist Figure 3In this context, 301 represents the core object of the event primitive, 302 represents the event type field, 303 represents the subject / object field, 304 represents the time field, 305 represents the spatial field, 306 represents the confidence level and initial risk value field, and 307 represents the data source, context, and rule label field.

[0029] exist Figure 4 In the diagram, 401 represents the input of the first source event primitive, 402 represents the input of the second source event primitive, 403 represents the input of the third source event primitive, 404 represents the alignment and association result, 405 represents the fusion result, and 406 represents the output of the target event primitive.

[0030] Figure reference numerals: 101 represents the multimodal data acquisition module; 102 represents the edge preprocessing module; 103 represents the event candidate extraction module; 104 represents the spatiotemporal alignment and subject association module; 105 represents the event primitive encoding module; 106 represents the multi-source fusion generation module; 107 represents the primitive output module; S101 represents the multimodal data acquisition step; S102 represents the edge preprocessing step; S103 represents the event candidate extraction step; S104 represents the spatiotemporal alignment and subject association step; S105 represents the event primitive encoding step; S106 represents the multi-source fusion generation module. The steps are as follows: S107 represents the primitive output step; 301 represents the core object of the event primitive; 302 represents the event type field; 303 represents the subject / object field; 304 represents the time field; 305 represents the spatial field; 306 represents the confidence level and initial risk value field; 307 represents the data source, context, and rule label field; 401 represents the first source event primitive input; 402 represents the second source event primitive input; 403 represents the third source event primitive input; 404 represents the alignment and association results; 405 represents the fusion result; and 406 represents the target event primitive output. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, but the scope of protection of the present invention is not limited to the following embodiments.

[0032] Example 1: Implementation of multimodal event primitive generation for hoisting operation scenarios.

[0033] In this embodiment, the smart engineering construction scenario is a large-scale hoisting operation area. The multimodal data acquisition module 101 is connected to a fixed camera, a tower crane controller, a UWB positioning tag, a hazardous gas sensor, and an operation log system. The fixed camera is used to acquire video streams of the hoisting operation area; the tower crane controller is used to output the boom rotation status, lifting weight status, and operating status; the UWB positioning tag is used to acquire personnel location and movement trajectory; the hazardous gas sensor is used to output environmental parameters; and the operation log system is used to output process number, operation area information, and operation permit information.

[0034] After receiving the aforementioned multimodal raw data, the edge preprocessing module 102 first standardizes the timestamp format, performs target detection and tracking on the video stream, performs trajectory smoothing on the positioning data, performs status standardization on the equipment logs, and performs outlier removal and threshold normalization on the environmental parameters. Then, the event candidate extraction module 103 extracts candidate events such as "personnel approaching the hoisting area" from the video, "hoisting equipment in operation" from the tower crane controller logs, "target personnel entering a danger zone" from the positioning data, and "current process is a hoisting process" from the work logs.

[0035] The spatiotemporal alignment and subject association module 104 maps the aforementioned candidate events to a unified scene time window and a unified spatial coordinate system. For candidate events of the same person in video and location data, subject matching is performed based on trajectory continuity and temporal overlap; for equipment operation events and hoisting area events, the operational relationship is associated based on the equipment number and hoisting process number. After processing, the event object to be encoded is obtained, which at least reflects the composite relationship between personnel, equipment, area, and process.

[0036] The event primitive encoding module 105 generates event primitives based on a unified event primitive structure. Taking the composite risk event of "personnel entering the hoisting hazard area while the equipment is in operation" as an example, event primitive EP1 can be generated, which includes: primitive number, event type "personnel-equipment hazard interaction event", event subject is the corresponding personnel identifier, event object is the hoisting equipment identifier, start time, end time, spatial location is the hoisting hazard area identifier and coordinate range, confidence level is the result of joint calculation of video and positioning, initial risk value is the result calculated based on the hoisting status and personnel distance, data source is video stream / positioning / UWB / equipment log, context label is hoisting procedure, and rule label is hoisting prohibition rule number.

[0037] After generating multiple event primitives, the multi-source fusion generation module 106 performs redundant merging of primitives describing the same event. For example, when both video stream and UWB positioning indicate that a person is in a dangerous area, the system merges the two into a unified target event primitive and recalculates the fusion confidence based on video recognition confidence and positioning accuracy. When there are inconsistencies between equipment logs and work logs regarding the current process status, the system performs conflict resolution based on rule priority and time consistency. After completing the above processing, the primitive output module 107 sends the target event primitive to the risk identification module and the early warning module for subsequent early warning intervention.

[0038] Example 2: A modified implementation method for high-altitude operation scenarios.

[0039] In this embodiment, the multimodal data acquisition module 101 connects to a camera in the high-altitude work area, a wearable terminal for workers, a positioning module, and a safety rope status detection module. The system extracts candidate events for "abnormal high-altitude work posture" from the video and wearable terminal, candidate events for "not attached or abnormally attached" from the safety rope status module, and candidate events for "staying in the edge area of ​​high altitude" from the positioning data. Standardized event primitives are generated through steps S104 and S105, and after fusion, composite risk event primitives are obtained, which are used to trigger high-level warnings.

[0040] Example 3: Adaptive expansion implementation of event primitive fields.

[0041] like Figure 3 As shown, the core object 301 of the event primitive can be implemented using a key-value pair structure, a table structure, a binary encoded structure, or a JSON structure. 302 to 307 represent different sets of fields. For standard scenarios, event primitives must retain at least the event type, subject / object, time, location, and confidence level fields; for complex scenarios, risk propagation chain fields, process constraint fields, environmental threshold fields, and rule association fields can be added based on the template. Through this structural design, the system can reuse a unified event primitive framework in different engineering scenarios while maintaining field extensibility.

[0042] Example 4: Implementation of multi-source fusion and low confidence compensation.

[0043] like Figure 4 As shown, the first source event primitive input 401, the second source event primitive input 402, and the third source event primitive input 403 are derived from video analysis results, device log analysis results, and environmental parameter analysis results, respectively. The system first completes time window overlap judgment, subject matching, and spatial relationship confirmation in the alignment and association results 404, and then performs redundancy elimination and conflict resolution in the fusion results 405. If the initial confidence of a source event primitive is low, but it is highly consistent with the other two high-confidence primitives in terms of subject, time, and space, the fusion module 106 compensates for the low-confidence primitive and generates the target event primitive output 406. If there is a significant conflict among the three and it cannot be resolved by rules, the system retains the conflict marker and reports it for manual review.

[0044] Example 5: Primitive compression upload and cloud-edge collaboration implementation.

[0045] In a preferred implementation, the primitive output module 107 performs compression encoding on the event primitive before sending it to the cloud. The compression method can be one or more of field trimming, binary packaging, template index replacement, and differential upload. For example, for similar event primitives that are repeatedly reported within a short period, only the changed fields and the timestamp difference are uploaded; for fixed fields in the scene template, the template index method is used to replace the repeated text fields. Compressed upload can further reduce network bandwidth usage and improve edge-cloud collaboration efficiency.

[0046] Furthermore, the candidate event extraction model in this invention is not limited to target detection models, behavior recognition models, and rule-triggered models, but can also be replaced by temporal action recognition models, transformer models, or multimodal joint representation models; the time alignment method is not limited to sliding time window aggregation, but can also adopt event-driven alignment or probabilistic graph-based alignment; the conflict resolution method is not limited to rule priority-based methods, but can also adopt Bayesian confidence update-based or graph model inference-based methods. All the above substitutions or equivalent solutions should fall within the protection scope of this invention.

Claims

1. A multi-modal event primitive extraction, encoding and collaborative generation system for smart engineering construction scenarios, characterized in that, It includes a multimodal data acquisition module, an edge preprocessing module, an event candidate extraction module, a spatiotemporal alignment module, a subject association module, an event primitive encoding module, a multi-source fusion generation module, and a primitive output module; The multimodal data acquisition module is used to collect multimodal raw data in the smart engineering construction scenario and send the multimodal raw data to the edge preprocessing module; The edge preprocessing module is used to perform one or more of the following processes on the multimodal raw data: timestamp normalization, data cleaning, noise suppression, target object extraction, behavior state extraction, device state standardization, and preliminary spatial coordinate mapping. The event candidate extraction module is used to extract event candidate fragments from preprocessed multimodal feature data according to preset triggering rules and / or recognition models; The spatiotemporal alignment module is used to perform temporal alignment and spatial mapping on event candidate fragments generated by different modalities; The subject association module is used to perform subject matching on the event candidate fragments after time alignment and spatial mapping to obtain the event object to be encoded under a unified subject entity; The event primitive encoding module is used to encode the event object to be encoded according to the unified event primitive structure, and generate standardized event primitives; The multi-source fusion generation module is used to perform redundancy elimination, conflict resolution, credibility correction and fusion output on event primitives generated by different modalities to generate target event primitives; The primitive output module is used to output the target event primitives to the risk identification module, knowledge graph module, rule reasoning module, early warning module and / or cloud storage module.

2. The system of claim 1, wherein, The multimodal raw data includes one or more of the following: video data, image data, audio data, environmental sensor data, device status data, positioning data, wearable terminal data, and operation log data.

3. The system of claim 1, wherein, The spatiotemporal alignment module includes a unified system clock correction unit, a sliding time window aggregation unit, and a spatial coordinate mapping unit, which are used to uniformly map candidate events generated by different acquisition devices to a unified coordinate system and a unified time axis of the engineering scene.

4. The system of claim 1, wherein, The subject association module performs subject matching on candidate event fragments from different modalities based on one or more of the following: personnel identification, equipment identification, trajectory continuity, spatial adjacency, job task number, and temporal overlap.

5. The system according to claim 1, characterized in that, The standardized event primitives generated by the event primitive encoding module include at least eight of the following: primitive number, event type field, event subject field, event object field, start time field, end time field, spatiotemporal location field, confidence level field, initial risk value field, data source field, context field, and rule label field.

6. The system according to claim 1, characterized in that, The multi-source fusion generation module performs redundancy elimination and conflict resolution on multiple event primitives based on event type consistency, subject consistency, spatiotemporal overlap, context similarity, and data source credibility, and outputs single-source event primitives or composite event primitives.

7. A method for extracting, encoding, and collaboratively generating multimodal event primitives for smart engineering construction scenarios, characterized in that, Includes the following steps: S101: Collect multimodal raw data in smart engineering construction scenarios; S102: Perform edge preprocessing on the original multimodal data to obtain multimodal feature data; S103: Extract event candidate fragments from the multimodal feature data according to preset triggering rules and / or recognition models; S104: Perform time alignment, spatial mapping, and subject association on the candidate event fragments to obtain the event object to be encoded; S105: Encode the event object to be encoded according to the unified event primitive structure to generate standardized event primitives; S106: Perform redundancy elimination, conflict resolution, and fusion on event primitives generated in different modalities to obtain the target event primitive; S107: Output the target event primitive to one or more of the risk identification, knowledge graph modeling, rule reasoning and early warning intervention modules.

8. The method according to claim 7, characterized in that, In step S102, the edge preprocessing includes one or more of the following: timestamp normalization, outlier correction, trajectory smoothing, target object extraction, behavior state extraction, device state standardization, and preliminary spatial coordinate mapping.

9. The method according to claim 7, characterized in that, In step S104, the time alignment uses a unified time window mechanism to aggregate candidate events of different modalities, the spatial mapping uses a scene registration matrix to map local coordinates to a unified scene coordinate system, and the subject association is matched based on entity identifier, trajectory continuity, and job relationship graph.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 7 to 9.

Images