Multi-modal risk evidence chain intelligent collection method for traffic engineering construction safety supervision

Abstract

The application provides a kind of multi-modal risk evidence chain intelligent collection method for traffic engineering construction safety supervision, belongs to the technical field of traffic engineering safety supervision and multi-modal data processing.To solve the problem that the existing engineering safety supervision data is scattered, multi-source heterogeneous data lacks unified alignment benchmark, and static single round verification lacks adaptability, the application constructs a risk evidence chain representation model with a four-tuple structure, establishes a three-anchor alignment benchmark of space, time and semantics, extracts multi-modal evidence units based on a preset time window centered on the trigger time of a risk event, constructs evidence correlation logic through cross-modal consistency verification, assesses the confidence level in combination with evidence completeness and conclusion consistency, triggers multi-round dynamic expansion verification when the confidence level is insufficient, and outputs a structured and traceable risk evidence chain.The application can effectively improve the automation level of engineering safety supervision and the objectivity of risk analysis.

Description

Technical Field

[0001] This invention belongs to the field of traffic engineering safety supervision and multimodal data processing technology, and relates to an intelligent collection method for multimodal risk evidence chains for traffic engineering construction safety supervision. Background Technology

[0002] In the construction of transportation engineering projects, there are often various high-risk work scenarios, such as deep foundation pits, high slopes, tunnel faces, and scaffolding erection. Current safety supervision of these scenarios faces prominent problems such as fragmented data, difficulty in corroborating evidence, and reliance on manual experience for risk assessment. Traditional safety inspection methods mainly rely on on-site inspections by management personnel and paper records. Data collection is fragmented, evidence organization lacks a structured framework, leading to strong subjectivity in risk assessment and difficulty in forming traceable, closed-loop qualitative conclusions.

[0003] With the application of information technology, engineering sites have deployed multi-source systems such as video surveillance, IoT sensors, inspection applications, engineering document libraries, and standard libraries. However, the data from these heterogeneous systems differ significantly in spatial positioning accuracy, time synchronization standards, and semantic tagging systems, lacking a unified reference system for accurate comparison. Furthermore, existing verification mechanisms are mostly static, single-round searches. When faced with insufficient or conflicting evidence, they lack dynamic mechanisms for supplementary evidence and quantitative confidence assessment, failing to effectively support automated and standardized security risk assessment and qualitative analysis. Therefore, there is an urgent need for an intelligent data collection method that can transform multi-source heterogeneous data into a computable and traceable chain of evidence. Summary of the Invention

[0004] To address the shortcomings and deficiencies of existing technologies, this invention provides an intelligent method for collecting multimodal risk evidence chains for safety supervision in transportation engineering construction. The method first constructs a four-tuple representation model of the multimodal risk evidence chain and establishes a triple anchor alignment benchmark encompassing space, time, and semantics. Spatial anchors utilize a dual positioning method combining work breakdown structure coding and geographic coordinates or measurement point numbers; time anchors employ timestamps of uniform precision; and semantic anchors adopt a standardized hazard labeling system, achieving a unified reference system mapping for multimodal heterogeneous data. Subsequently, in response to the triggering of a risk event, the corresponding spatial and temporal anchors are extracted to determine the multimodal data source. Centered on the risk event trigger time and based on a preset time window corresponding to the risk type, raw data is acquired from various multimodal data sources, and feature extraction and evidence anchor generation are performed. This data is mapped into standardized evidence units and aggregated to form an evidence unit set. Next, cross-modal consistency verification is performed on the evidence units to determine the association type and construct an association logic set. Finally, a comprehensive confidence score is calculated based on the completeness of evidence and the consistency of conclusions, and a confidence level is mapped. If the confidence level indicates that supplementary evidence is needed, a multi-round dynamic expansion verification mechanism is triggered. This mechanism expands the spatial retrieval scope and time sliding window to obtain supplementary evidence to update the evidence unit set and the related logic set, and recalculates the confidence level until the confidence level no longer indicates that supplementary evidence is needed or the maximum number of iterations is reached. The final multimodal risk evidence chain is then output. This method generates the evidence chain as a computable object and ensures the completeness and reliability of the evidence through a dynamic verification closed loop.

[0005] The specific technical solution adopted by this invention to solve its technical problem is as follows:

[0006] This invention aims to solve the technical problems of lack of unified alignment benchmarks for multi-source heterogeneous data, missing evidence association logic, and insufficient adaptability of static verification in the existing safety supervision of transportation engineering construction. It provides a method and system for intelligent collection of multimodal risk evidence chains for safety supervision of transportation engineering construction.

[0007] To achieve the above objectives, the present invention adopts the following technical solution:

[0008] This invention first provides an intelligent collection method for multimodal risk evidence chains for safety supervision in transportation engineering construction, comprising: constructing a four-tuple representation model of the multimodal risk evidence chain, wherein the four-tuple includes a risk event node, a set of evidence units, a set of related logics, and a confidence level assessment value; establishing a triple anchor point alignment benchmark of space, time, and semantics, wherein the spatial anchor point adopts dual positioning using work breakdown structure coding and geographic coordinates or measurement point numbers to adapt to the scenario characteristics of transportation engineering where construction organization division and physical coordinates coexist; the time anchor point adopts a timestamp of uniform precision, and various business systems are synchronized through a network time protocol; the semantic anchor point adopts a standardized hazard labeling system to realize the mapping of multimodal heterogeneous data under a unified reference system.

[0009] In response to the triggering of a risk event, the spatial and temporal anchor points corresponding to the risk event are extracted to determine the multimodal data source within the corresponding spatiotemporal range.

[0010] Centered on the trigger time of the risk event, and based on the preset time window of the corresponding risk type, the raw data within the time window range is obtained from various multimodal data sources. Feature extraction and evidence anchor point generation are performed on the raw data, which are then mapped into standardized evidence units and summarized to form the evidence unit set.

[0011] Perform cross-modal consistency verification on the evidence units in the evidence unit set to determine the relationship type between each pair of evidence units and construct the association logic set.

[0012] A comprehensive confidence score is calculated based on the completeness of the evidence unit set and the consistency of the conclusions of the related logic set, and mapped to the corresponding confidence level. If the confidence level is to be supplemented, a multi-round dynamic expansion verification mechanism is triggered to expand the search scope to obtain supplementary evidence to update the evidence unit set and the related logic set, and the confidence level is recalculated until the confidence level is no longer to be supplemented or the maximum number of iterations is reached, and the final multimodal risk evidence chain is output.

[0013] Furthermore, each evidence unit is a five-tuple structure, including unit identifier, evidence type, evidence content, evidence anchor information, and collection timestamp; the evidence types include on-site images, IoT monitoring, inspection logs, engineering documents, standard provisions, business data, and scenario context; the relationship types include supporting relationships, supplementary relationships, and conflicting relationships. The output multimodal risk evidence chain satisfies the feasible evidence chain constraint: the total number of evidence units is not less than three, all evidence units contain non-empty evidence anchor information, and the evidence unit set simultaneously includes on-site image evidence, IoT monitoring or business data evidence, and standard provisions or engineering document evidence.

[0014] Furthermore, the cross-modal consistency verification adopts corresponding verification rules based on the evidence type combination, specifically including: for the evidence combination of on-site images and IoT monitoring, calculating the visual anomaly degree of the images and the anomaly degree of the IoT monitoring respectively; if the absolute value of the difference between the two is not greater than a preset threshold, it is determined to be a supporting relationship; otherwise, it is determined to be a conflicting relationship; for upper limit type indicators, the IoT monitoring anomaly degree is calculated based on the indicator risk direction, warning value, alarm value, and rate of change, and the calculation formula is:

[0015]

[0016] In the formula, These are actual measured values ​​from IoT sensors. The indicator's warning threshold, The indicator alarm threshold, This quantifies the degree of indicator jump. For the evidence combination of regulatory provisions and on-site images or IoT monitoring, the regulatory provisions are parsed into a constraint set containing predicates, objects, and limits. The on-site evidence is converted into a feature predicate set. First, a risk proposition H to be verified is defined. If the constraint set matches the semantics of the on-site features and the on-site value violates the regulatory limits, then the regulatory evidence supports the risk proposition H. If the on-site value meets the regulatory limits, then a rebuttal relationship is formed (a rebuttal relationship is a type of conflict relationship). No association is established when the provisions are inapplicable or semantically mismatched. The semantic matching can be achieved by calculating the cosine similarity of the semantic embedding vectors and setting a corresponding threshold. For the evidence combination of engineering documents and on-site images or IoT monitoring, the consistency of the work points and monitoring indicators of the engineering documents and on-site evidence is verified. If they are consistent, a supplementary relationship is determined; otherwise, no association is established.

[0017] Furthermore, the preset time windows are set according to risk type, with a pre-event window Tpre and a post-event window Tpost, and corresponding scenario-based adjustment rules are set; "±Th" means Tpre=Tpost=Th: the basic time window for deep foundation pit risk is ±24h. If the time consistency deviation is greater than the basic time window and the IoT sampling frequency is not less than 1 time / h, the time window is expanded by 12h, with a maximum of 48h; the basic time window for tunnel risk is ±12h, the time window for gas-related risks is shortened to ±6h, and the time window for abnormal ventilation scenarios is fixed at ±3h; the basic time window for scaffolding risk is ±8h. If only image triggers and there is no IoT data, the time window is increased by 4h based on the baseline value; the basic time window for high slope risk is ±24h. When the daily rainfall is greater than 25mm / d, the time window is expanded to ±36h; the basic time window for edge protection risk is ±4h, and the time window for events triggered by manual reporting is shortened to ±2h. Furthermore, the time window is dynamically adjusted using a general adaptive rule. The adjusted time window is the smaller of the following two values: the base time window of the current round multiplied by one, plus a preset coefficient and the product of the number of missing key evidence types, and the maximum upper limit of the time window. As a preferred implementation, the preset coefficient can be 0.1. The adjusted time window satisfies the following relationship: In the formula, This serves as the base time window for the current round. This is the maximum upper limit of the time window. This represents the number of key evidence types missing during the initial alignment.

[0018] Furthermore, the maximum number of iterations of the multi-round dynamic expansion verification mechanism is a preset value, preferably 3 times; each expansion simultaneously performs spatial breadth expansion and temporal depth expansion. Spatially, it prioritizes expansion along adjacent nodes of WBS, the same structural unit, the same monitoring section, and the camera field of view coverage relationship. If necessary, it expands the search range according to the configurable physical radius; temporally, it expands the time sliding window increment, preferably by 1 hour per round of backtracking expansion; after updating the spatiotemporal boundaries, the evidence extraction, association construction, and confidence assessment processes are re-executed.

[0019] Furthermore, the comprehensive confidence score is calculated by weighting the evidence completeness score and the conclusion consistency score. The evidence completeness score is determined based on sub-scores across five dimensions: image evidence, IoT evidence, normative matching, anchor points, and logical connections. The conclusion consistency score is the average of all quantifiable evidence against the consistency metric. Based on the comprehensive confidence score, the confidence level is mapped to four levels: high confidence, medium confidence, low confidence, and need for supplementary evidence. The contribution rules of relationships to the confidence score are as follows: supporting relationships contribute positively to the conclusion consistency score calculation; supplementary relationships improve the evidence completeness score but do not participate in conflict detection; conflicting relationships trigger IoT data re-collection or time window expansion for supplementary evidence; if conflicts still exist, the case is transferred to manual review and the confidence level is lowered by one level.

[0020] Furthermore, when the confidence level is "pending further verification," a supplementary verification task is generated and dispatched to the retrieval module or on-site inspection terminal. The supplementary verification task includes a task identifier, a risk event identifier, a conflict or supplementary verification type, a list of key evidence types to be supplemented, extended time window parameters, the responsible person, and a completion deadline. If the supplementary verification task is not completed within the deadline, it is escalated to the security administrator. Two timeouts will forcibly set the evidence chain confidence level to "pending further verification," and the automatic handling process for that risk event will be blocked.

[0021] Furthermore, the evidence anchor information includes the source system identifier, collection timestamp, file number, standard clause number, document page number, and data hash value; structured data uses a secure hash algorithm to calculate the hash of the standardized JSON content after sorting and removing whitespace from the fields; for image, video, and other file-type evidence, the hash is calculated by combining the original file binary, source handle, and key metadata. Multimodal data sources are encapsulated into a unified calling interface through model context protocols or function call specifications, allowing agents to obtain data in a low-intrusive manner, and uniformly mapped into standardized JSON format evidence units. During evidence structuring, spatiotemporal subgraph data such as engineering structure parameters, historical risk records, and equipment location attributes within the corresponding spatiotemporal range are extracted from the dynamically configurable knowledge graph based on the spatiotemporal anchor points of risk events, and incorporated into scene context-type evidence units to supplement relationships and integrate them into the evidence chain.

[0022] Furthermore, the feature extraction methods for raw data of different modalities vary depending on the modality type: for image data, target detection and scene classification models are used to extract targets, labels and severity scores; for IoT data, sliding window statistics are used to calculate the mean, slope or number of out-of-bounds occurrences; for text data, named entity recognition is used to extract risk entities and constraint predicates.

[0023] Corresponding to the above method, the present invention also provides a multimodal risk evidence chain intelligent collection system for safety supervision of transportation engineering construction, comprising:

[0024] The anchor configuration module is used to construct a quadruple representation model of a multimodal risk evidence chain and establish a triple anchor alignment benchmark of space, time and semantics. The spatial anchors are located by dual positioning using work breakdown structure coding and geographic coordinates or measurement point numbers.

[0025] The event triggering module is used to receive risk event triggering information, extract the corresponding spatiotemporal anchor point, and determine the multimodal data source within the corresponding spatiotemporal range;

[0026] The evidence extraction module is used to obtain raw data from various multimodal data sources based on a preset time window, with the trigger time of the risk event as the center, generate standardized evidence units, and summarize them to form an evidence unit set.

[0027] The association construction module is used to perform cross-modal consistency verification on evidence units, determine the type of association relationship between evidence units, and construct an association logic set.

[0028] The confidence assessment module is used to calculate a comprehensive confidence score based on the completeness of evidence and the consistency of conclusions and to map the confidence level. If the confidence level is to be supplemented, a multi-round dynamic expansion verification mechanism is triggered to update the evidence chain until the confidence level is no longer to be supplemented or the maximum number of iterations is reached, and the final multimodal risk evidence chain is output.

[0029] Furthermore, the system also includes a dynamic expansion and supplementary certification management module, which is used to trigger a multi-round dynamic expansion and verification mechanism when the confidence level is pending supplementary certification, generate and dispatch supplementary certification tasks, and manage the supplementary certification timeout upgrade process and automatic handling blocking logic.

[0030] In the above technical solution, the preset parameters such as the base value of the time window, the adjustment coefficient, the number of iterations, the search radius, the time sliding window increment, and the threshold are all preferred implementation methods. In actual applications, they can be adaptively adjusted according to engineering scenarios and regulatory requirements.

[0031] Compared to existing technologies, this invention and its preferred solutions offer at least the following advantages: First, by defining a four-tuple evidence chain representation model and a triple anchor alignment benchmark, multi-source heterogeneous data is transformed into computable objects under a unified reference system, solving the problems of data silos and spatiotemporal alignment difficulties in traffic engineering scenarios, and realizing the structured and traceable organization of evidence. Second, through cross-modal consistency verification rules, the normative clauses are parsed into a set of constraints and semantically matched with on-site features, achieving quantitative cross-verification between images, IoT monitoring, and normative clauses, improving the objectivity and automation level of evidence association. Third, through scenario-based adaptive time window adjustment rules and a multi-round dynamic expansion verification mechanism, the invention effectively addresses situations of evidence loss and evidence conflict in complex engineering environments, improving the coverage and robustness of verification, and overcoming the limitations of static single-round verification. Finally, by combining confidence scoring assessment and a mechanism to downgrade and block supplementary evidence tasks, a complete closed loop from data collection to risk assessment was constructed. This ensures that administrative closed-loop intervention can be triggered when the confidence level of the evidence chain is insufficient, avoiding misjudgment and blind handling of risks due to insufficient evidence, and providing a reliable decision-making basis for automated safety supervision. Attached Figure Description

[0032] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0033] Figure 1 This is a diagram illustrating the overall technical architecture of the multimodal risk evidence chain intelligent agent upon which this invention is based. Detailed Implementation

[0034] To make the features and advantages of the present invention more apparent and understandable, specific embodiments are described below in detail:

[0035] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used in this specification have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0036] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0037] This invention aims to provide an intelligent method for collecting multimodal risk evidence chains for safety supervision in transportation engineering construction. Based on the principles of evidence-driven, traceable, and low-intrusion, it uses existing business systems as data sources and intelligent agents as evidence organizers to achieve intelligent generation of structured evidence loops from fragmented data collection. This invention constructs a multimodal risk evidence chain representation model, establishes a spatiotemporal semantic triple anchor point alignment mechanism, and employs a time window-based alignment generation algorithm and a multi-round dynamic expansion verification mechanism to automatically generate structured, traceable evidence chains with quantifiable confidence levels. This addresses the technical problems of lacking a unified alignment benchmark for multimodal data, missing evidence association logic, and insufficient adaptability for static verification.

[0038] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 1 As shown, the multimodal risk evidence chain intelligent agent on which this embodiment relies adopts a layered overall technical architecture, including a multimodal perception access layer, a knowledge retrieval enhancement layer, a multi-agent reasoning layer, a risk evidence chain generation layer, and a manual review and rectification closed-loop layer. The right side of the architecture configures the tool to call external system modules, and achieves low-intrusion docking with existing heterogeneous systems through the model context protocol and Function Calling mechanism. The multimodal perception access layer aggregates multi-source data such as video surveillance, drone images, IoT sensors, inspection logs, construction plans, geological survey data, standards and regulations, and business systems, forming the original data source for the evidence chain. The knowledge retrieval enhancement layer integrates the capabilities of standard clause retrieval, engineering document retrieval, historical case retrieval, multimodal vector retrieval, and unified indexing of risk events, providing a knowledge foundation for triple anchor alignment, semantic matching, and evidence association. The multi-agent reasoning layer sequentially executes task decomposition, evidence collection and invocation, evidence verification and conflict identification, confidence assessment, and supplementary evidence suggestions, serving as the execution carrier for the TW-ECGA evidence chain generation algorithm and multi-round dynamic expansion verification mechanism in this embodiment. The risk evidence chain generation layer outputs structured evidence chain results, covering risk event number, evidence source and anchor point, association logic, standard basis, risk level, and handling suggestions. The manual review and rectification closed-loop layer carries the functions of manual confirmation / rejection / supplementary evidence, draft work orders, rectification closed loop, review record keeping, and audit traceability, realizing full-process traceable management of the evidence chain from generation to implementation.

[0040] The multimodal risk evidence chain intelligent acquisition method proposed in this invention first defines a structured evidence chain representation model. The multimodal risk evidence chain EChain is represented as a four-tuple, including a risk event node ev, an evidence unit set ESet, an association logic set LSet, and a confidence level assessment value conf. Specifically, the risk event ev is defined as a six-tuple, containing a unique risk event identifier evId, a risk type evType, an engineering object or monitoring point evObj, a timestamp of occurrence or triggering evTime, a work site location or mileage marker evLoc, and a textual description of the abnormal behavior evDesc. The evidence unit ei is defined as a five-tuple, containing a unique evidence identifier eiId, an evidence type eiType, an evidence content summary or structured feature eiContent, evidence anchor information eiAnchor, and an evidence acquisition timestamp eiTime. Evidence types cover various modalities such as on-site images, IoT monitoring, inspection logs, engineering documents, specification clauses, and business data. The association logic lj is defined as a triple, which includes the source evidence unit ei_s, the target evidence unit ei_t, and the association relationship type relType. Among them, relType includes support (mutual support), supplement (background supplementation), and conflict (conflict).

[0041] To ensure accurate comparison of multimodal data within the same reference frame, this invention establishes a triple anchor alignment matrix. For spatial anchors, a dual positioning system is employed, combining Work Breakdown Structure (WBS) coding with GPS coordinates or measurement point numbers, to adapt to the characteristics of traffic engineering scenarios that involve both construction organization divisions and physical coordinates. For temporal anchors, microsecond-level timestamps are uniformly used, and all business systems are required to synchronize via Network Time Protocol (NTP). For semantic anchors, a standardized hazard labeling system is adopted, such as seepage, settlement, and cracking, to eliminate differences in semantic expression between multi-source heterogeneous data. The evidence anchor `eiAnchor` records detailed information such as the source system identifier, collection timestamp, file number, specification clause number, document page number, and data hash value. Its generation rules are based on direct acquisition of fields such as the data source system identifier, business record timestamp, and business number, or calculated using a secure hash algorithm. For example, the data hash value is calculated using the SHA-256 algorithm on the normalized JSON content after sorting and removing whitespace from the field names, ensuring the traceability and immutability of the evidence source.

[0042] The evidence chain construction process mainly includes four steps: multimodal data access, evidence structuring, evidence chain organization, and confidence level assessment, and is integrated with the spatiotemporal subgraph extraction of the Dynamically Configurable Knowledge Graph (DCKG). In the multimodal data access phase, the system receives data from videos or images, IoT sensors (typically with a default sampling frequency of 5 minutes / time), inspection logs, engineering documents, standards and specifications, and business systems. In the evidence structuring phase, based on the spatial and temporal anchor points of risk events, the system extracts spatiotemporal subgraph data such as engineering structure parameters, historical risk records, and equipment location attributes within the corresponding spatiotemporal range from the DCKG, categorizing them into scenario context-based evidence units and incorporating them into the evidence chain using supplementary association logic. Simultaneously, feature extraction, timestamp alignment, risk labeling, and evidence anchor point generation are performed on data from different modalities. As a preferred implementation, image feature extraction can employ object detection and scene classification models such as YOLOv8 or ResNet, outputting content including the target, label, and severity score; IoT numerical features utilize sliding window statistics to calculate the mean, slope, or number of out-of-bounds occurrences; text features are extracted using a named entity recognition model to extract risk entities and constraint predicates. Addressing the issue of heterogeneous interfaces in existing traffic engineering systems, as a preferred implementation, query interfaces from different data sources can be encapsulated into a unified tool using the Model Context Protocol (MCP) or Function Calling specifications. This tool can be invoked by the agent to retrieve data, and the aforementioned multi-source heterogeneous data can be uniformly mapped into JSON evidence units conforming to a five-tuple structure, achieving low-intrusion data access and semantic fusion.

[0043] During the evidence chain organization phase, the system organizes traceable objects according to risk events, evidence sources, correlation logic, and confidence levels. Risk events are triggered by IoT alarm pushes, video analysis outputs, and manual reports; these three types of events are uniformly written into the event bus and generate a six-tuple of risk events. When an event is triggered, the system locks the spatiotemporal boundaries, links the camera asset table and sensor asset table to lock surrounding sensing devices, and enters a multimodal parallel acquisition process. Subsequently, the Large Language Model (LLM) performs semantic fusion on the multi-source JSON evidence units, verifying spatiotemporal and causal alignment relationships, such as forming a closed-loop qualitative conclusion by combining wet stain images, accelerated settlement IoT data, and inspection crack records.

[0044] To transform evidence chain generation into a solvable mathematical problem, this invention proposes an evidence chain generation problem model for anomaly detection. First, a feasible evidence chain must satisfy the following constraints: the size of the evidence unit set must be greater than or equal to 3; all evidence units must have non-empty evidence anchors; and the set must simultaneously contain on-site image evidence, IoT monitoring or business data evidence, and regulatory clauses or engineering documents. Second, the temporal consistency deviation of the evidence chain is defined as the absolute value of the maximum deviation between the time of each piece of evidence collection and the time of the risk event, which is expressed as:

[0045]

[0046] The deviation must not exceed the event trigger time window. The completeness of the evidence chain is defined as the weighted coverage of validly established edges among the necessary links for each risk type, and its formula is expressed as:

[0047]

[0048] Based on the above definition, the multimodal risk evidence chain generation problem for anomaly detection is modeled as a multi-objective optimization problem. Its optimization objective is to minimize temporal consistency deviation and maximize the required association coverage while satisfying the constraints of necessary evidence slots and associated edges for each risk type. This can be expressed as:

[0049]

[0050] As a preferred implementation, this multi-objective optimization problem can be transformed into a single objective by using the linear weighted summation method. The time consistency weight and the necessary association coverage weight can be dynamically configured according to the risk type, and the target evidence chain that meets the constraints can be output through the candidate evidence screening rules.

[0051] Based on the aforementioned problem model, this invention proposes a master algorithm for evidence chain generation based on time window alignment, TW-ECGA. This algorithm considers the risk event (ev) and the time window size... The TW-ECGA algorithm takes a multimodal data source set DS as input and outputs a target evidence chain EChain that satisfies preset constraints. First, it initializes the evidence unit set ESet and the association logic set LSet to empty. Then, for each data source ds in DS, the time window alignment sub-algorithm TWA is called in parallel. The TWA sub-algorithm takes the risk event ev and the time window size as inputs. With a single data source ds as input, based on the risk event timestamp evTime and time window The algorithm calculates the start and end time boundaries, queries data records within the corresponding time range from the data source, extracts features and generates anchor points for each record, forming an evidence subset which is then returned. The TW-ECGA algorithm collects evidence units returned from each data source and adds them to the total evidence unit set. It then calls the correlation logic construction sub-algorithm LCA to generate a correlation logic set. The LCA sub-algorithm iterates through all non-repeating evidence pairs in the evidence unit set and calls the corresponding consistency verification function based on the evidence type. Finally, the TW-ECGA algorithm calls the confidence assessment function to calculate the comprehensive confidence score. If the assessment result indicates that supplementary evidence is needed, a supplementary evidence request and multiple rounds of dynamic expansion are triggered. The specific algorithm pseudocode is as follows:

[0052] Algorithm 1: TW-ECGA Main Algorithm for Evidence Chain Generation Based on Time Window Alignment

[0053] Input: Risk event ev, time window size T_w, multimodal data source set DS

[0054] Output: The target evidence chain EChain* that satisfies the preset constraints.

[0055] 1.ESet←empty set;LSet←empty set

[0056] 2. For each data source ds in DS, execute / / in parallel.

[0057] 3. Using ev, T_w, and ds as inputs, call the time window alignment algorithm TWA.

[0058] 4. Add the returned evidence unit to ESet

[0059] 5. End of discussion

[0060] 6. Using ESet as input, invoke the associative logic to construct the sub-algorithm LCA.

[0061] 7.conf ← Calls the confidence evaluation function (ESet, LSet)

[0062] 8.EChain*←(ev,ESet,LSet,conf)

[0063] 9. If conf == "Pending supplementary certification", then a supplementary certification request and multiple rounds of expansion will be triggered.

[0064] 10. Output EChain*

[0065] Algorithm 2: Time Window Alignment Sub-algorithm TWA

[0066] Input: Risk event ev, time window size T_w, data source ds

[0067] Output: Evidence unit set ESet_ds

[0068] 1. ESet_ds ← Empty set

[0069] 2.timeStart←evTime-T_w; timeEnd←evTime+T_w

[0070] 3. records ← Retrieves data records from ds within the time range from timeStart to timeEnd.

[0071] 4. For each record in records, execute the following:

[0072] 5. ei ← Perform feature extraction and anchor point generation on rec

[0073] 6. Add ei to ESet_ds

[0074] 7. End of section

[0075] 8. Output ESet_ds

[0076] Algorithm 3: Associative Logic Construction Sub-algorithm LCA

[0077] Input: Evidence unit set ESet

[0078] Output: Associative logical set LSet

[0079] 1.LSet←empty set

[0080] 2. For each pair of unique evidence units (ei_i, ei_j) in ESet, perform the following:

[0081] 3. If ei_i.eiType == live video and ei_j.eiType == IoT monitoring, then call matchImgIoT.

[0082] 4. If they match, add a supporting relationship; otherwise, add a conflicting relationship.

[0083] 5. If ei_i.eiType == the specification clause and ei_j.eiType in {field image, IoT monitoring}, then call matchReg.

[0084] 6. If ei_i.eiType == project document and ei_j.eiType in {field image, IoT monitoring}, then call matchDoc.

[0085] 7. End of section

[0086] 8. Output LSet

[0087] The consistency verification function is the core of building the association logic. The matchImgIoT function calculates the visual anomaly degree of the image. and IoT monitoring anomaly Perform cross-validation. Values ​​normalized to For the interval, the anomaly rate of IoT monitoring is calculated based on the indicator risk direction, warning value, alarm value, and rate of change; for upper limit indicators, the calculation formula is as follows:

[0088]

[0089] in, These are actual measured values ​​from IoT sensors. These are the control or alarm thresholds for the monitored indicators. If the absolute value of the difference between the two satisfies... If the condition is met, the relationship is determined to be supportive; otherwise, it is determined to be conflictive. The matchReg function parses the specification clauses into a set of constraints containing predicates, objects, and limits, converts the field features into a set of predicates, and predefines the risk proposition H to be verified.

[0090] As a further preferred solution, for upper limit indicators, the anomaly degree of IoT monitoring is calculated based on the indicator's risk direction, warning value, alarm value, and rate of change. The calculation formula is as follows:

[0091]

[0092] In the formula, These are actual measured values ​​from IoT sensors. The indicator's warning threshold, The indicator alarm threshold, The quantitative value for the degree of indicator jump can be calculated by dividing the difference between the data at the current time and the previous time by the standard deviation of the historical time period.

[0093] As a preferred implementation, a pre-trained language model can be used to generate semantic embedding vectors. Cosine similarity is used as a matching metric. A semantic match is determined when the cosine similarity between the normative constraint predicate and the on-site feature predicate is greater than or equal to 0.7. This threshold can be optimized based on labeled samples from engineering scenarios and adjusted according to the strictness of regulations. If the constraint predicate semantically matches the on-site feature and the on-site value violates the normative limit, the normative evidence forms a support relationship for the risk proposition H; if the on-site value meets the normative limit, a refute relationship is formed; if the provision is inapplicable or the semantic matching is insufficient, no matching relationship is determined, and the document is not included in the association logic set. The `matchDoc` function verifies whether the work points in the engineering document and the on-site evidence are consistent and whether the monitoring indicators have the same name. If they are consistent, a supplement relationship is established; otherwise, no association is established.

[0094] The above thresholds are all preferred empirical values ​​under the scenario of traffic engineering safety supervision, and can be dynamically optimized according to the actual supervision accuracy requirements and the training results of scenario-labeled samples.

[0095] In view of the characteristics of different risk types, this invention sets recommended values ​​and adaptive adjustment rules for the window Tpre before the event occurs and the window Tpost after the event occurs; "±Th" means Tpre=Tpost=Th. For example, deep foundation pit recommendations for Hours, the tunnel is Hours, scaffolding Hours, high slope is Hours, edge protection is Hours. Meanwhile, condition adjustment rules were set for specific scenarios: for deep foundation pits, if the time consistency deviation is greater than [a certain value], [the rules will be adjusted accordingly]. And if the IoT sampling frequency is greater than or equal to once per hour, then The time window for gas-related risks is extended by 12 hours, but not exceeding 48 hours; for tunnels, the time window is shortened to [missing information]. For ventilation abnormalities, the timeframe is fixed at 6 hours; for scaffolding, if only image triggering occurs without IoT data, then... Add 4 hours to the baseline value; for high slopes, the daily rainfall... (Unit: mm / d) When the value is greater than 25, the time window is extended to For edge protection, the time window for manually reported events can be shortened to [number] hours; Hours. To provide general adaptive capabilities, this invention designs a general adaptive rule, with an adjusted time window. The calculation formula is:

[0096]

[0097] in, This is the maximum allowed value for the time window. The key evidence type is the number of missing key evidence types during the initial alignment. Key evidence types are determined by the required evidence slot table corresponding to the current risk type, and the count increments by 1 for each missing required slot. This rule allows the time window to be continuously and quantitatively adjusted based on the lack of evidence. When the initial evidence is insufficient to support a conclusion, the system automatically triggers a multi-round dynamic expansion verification mechanism, with a maximum of 3 iterations. The expansion strategy employs both spatial breadth and temporal depth. Spatially, it prioritizes expansion along adjacent nodes of the WBS, the same structural unit, the same monitoring section, and camera field of view coverage relationships, and expands according to the configurable physical radius when necessary. Temporally, it prioritizes increasing the time window increment. For example, backtracking by 1 hour. After updating the spatiotemporal boundaries, the system re-executes the multimodal parallel acquisition and subsequent processes. As a further preferred implementation, the time window increment adopts a bidirectional symmetrical expansion method, that is, the preceding and following boundaries of the time window are simultaneously increased by the corresponding duration; alternatively, a unidirectional backtracking expansion method can be adopted according to the risk type, expanding only in the historical direction before the event occurred.

[0098] Confidence level assessment is based on two dimensions: completeness of evidence and consistency of conclusions. Completeness of evidence scoring. The calculation formula is:

[0099]

[0100] The scoring criteria for the preferred implementation method are: completeness of image evidence. One point is awarded if there is spatiotemporally aligned on-site footage; otherwise, a point is awarded if footage exists but the time deviation exceeds [a certain threshold]. 0.5 points are awarded when 80% of the evidence is present, and 0 points are awarded when there is no image; IoT evidence completeness. 1 point is awarded for data with corresponding measurement points and time alignment, 0.5 points for discontinuous data, and 0 points for no corresponding data; Standard matching completeness. 1 point is awarded for matching a valid regulatory provision corresponding to the risk type, 0.5 points are awarded for matching only a general provision, and 0 points are awarded for no matching provision; Anchor point completeness One point is awarded when all six fields of the evidence anchor are complete, 0.5 points are awarded when one or two non-critical fields are missing, and 0 points are awarded when a critical field is missing. The completeness of logical association is calculated based on the weighted coverage rate of the required association edges. One point is awarded when the coverage rate is greater than or equal to 0.6, 0.5 points are awarded when it is between 0.3 and 0.6, and 0 points are awarded when it is less than 0.3. Conclusion Consistency Score For quantifiable evidence The mean. As a further preferred implementation, a consistency measure of single-pair evidence. The following rules apply: For quantifiable evidence pairs consisting of on-site video and IoT monitoring, the consistency metric is set to [value missing]. For evidence pairs consisting of regulatory provisions and on-site evidence, the consistency metric is set to 1 when the relationship is deemed supportive and 0 when the relationship is deemed conflictive.

[0101] Overall Confidence Score The calculation formula is:

[0102]

[0103] The system according to Values ​​are mapped hierarchically. For high confidence, For Zhongzhixin, For low confidence, Further evidence is needed. The semantics of the association have a differentiated impact on the confidence level; the support relationship is involved. The positive contribution of the supplement relationship is that it does not participate in conflict detection but enhances [the ability to detect conflicts]. Conflicts will trigger priority re-acquisition of IoT data or supplementary images within an extended time window. If conflicts persist, the data will be transferred to manual review and the confidence level will be lowered by one level. As a further preferred implementation, if the conflict is caused by instantaneous jumps in IoT data or single-point out-of-bounds errors, IoT data re-acquisition will be triggered to perform secondary verification on the original sampled data during the abnormal period. If the conflict is caused by large deviations in evidence collection time or incomplete time window coverage, time window extension will be triggered to supplement evidence.

[0104] To achieve a reliable administrative closed loop for the evidence chain, this invention designs a supplementary evidence task mechanism. When the confidence level of the evidence chain is "needs supplementary evidence," the system generates a supplementary evidence task, SupTask, which includes a task identifier (supId), a risk event identifier (evId), a conflict or supplementary evidence type (cType), a list of key evidence types to be supplemented (missingTypes[]), an extended time window parameter (timeWindowExt), the responsible person (assignee), and a completion deadline. The supplementary evidence task is sent to the application terminal of the retrieval agent or the on-site inspector. If the task is not completed within the deadline, it is escalated to the security administrator; if two timeouts occur, the confidence level of the evidence chain is forcibly set to "needs supplementary evidence," and the automatic handling process for the risk event is blocked to ensure the reliability of risk intervention.

[0105] The following is a specific test example: when an IoT inclination measurement out-of-bounds alarm occurs at a certain deep foundation pit location, the system receives the alarm and triggers a risk event, setting an initial time window. for Hours. The first round of investigation revealed missing scene footage; the number of missing key evidence types was calculated. Set to 1, and according to the adaptive rule, The time window was adjusted to 26.4 hours. Within the adjusted time window, the system expanded its search along adjacent nodes of the current WBS, the same monitoring section, and the camera field of view coverage relationship, retrieving monitoring frames containing wet stains from camera assets spatially associated with the target work site. During the semantic fusion stage, image feature extraction outputs visual anomaly levels. IoT programmatic parsing output monitoring anomaly degree The `matchImgIoT` function calculates that the difference between the two is less than 0.3, indicating a support relationship. Simultaneously, the system calls the `matchReg` function to semantically match the on-site wet stain characteristics with relevant clauses in the Technical Specification for Foundation Pit Support, verifying the semantic match between the on-site characteristics and the specification clauses, and determining whether they violate specification limits, thus forming supporting or rebuttal evidence for the risk proposition to be verified. Ultimately, the evidence chain includes three types of evidence: images, IoT, and specification clauses. The set of associated logic includes the support relationships between on-site facts and the support or rebuttal relationships between specification evidence and the risk proposition. The completeness of the evidence is calculated. The consistency is 0.9. The overall score is 0.75. The value is 0.825, which is considered high confidence, and a traceable closed-loop evidence chain is successfully output.

[0106] In summary, this invention offers the following advantages: First, by defining a four-tuple evidence chain representation model and a triple anchor alignment matrix, multi-source heterogeneous data is transformed into computable objects under a unified reference system, solving the problems of data silos and alignment difficulties in traffic engineering scenarios, and achieving structured and traceable organization of evidence. Second, by modeling evidence chain generation as a multi-objective optimization problem, and proposing the TW-ECGA algorithm and its supporting TWA and LCA sub-algorithms, efficient parallel acquisition of evidence and automated construction of related logic are achieved, improving the objectivity and efficiency of evidence chain generation. Third, through scenario-based adaptive time window adjustment rules and a multi-round dynamic expansion verification mechanism, the invention effectively addresses situations of missing and conflicting evidence in complex engineering environments, improving the coverage and robustness of verification. Finally, through quantitative confidence level assessment and a mechanism for downgrading and blocking supplementary evidence tasks, a complete closed loop from data collection to risk assessment is constructed, providing a reliable decision-making basis for automated safety supervision and avoiding risk misjudgment and blind handling due to insufficient evidence.

[0107] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A multimodal risk evidence chain intelligent collection method for safety supervision of transportation engineering construction, characterized in that, Includes the following steps: A multimodal risk evidence chain is constructed, including a quadruple representation model of risk event nodes, evidence unit set, related logic set, and confidence level assessment value. A spatial, temporal, and semantic triple anchor alignment benchmark is established. The spatial anchor uses dual positioning with work breakdown structure coding and geographic coordinates or measurement point number. The temporal anchor uses a timestamp with uniform precision. The semantic anchor uses a standardized hazard labeling system to achieve a unified reference system mapping for multimodal heterogeneous data. In response to the triggering of a risk event, the spatial and temporal anchor points corresponding to the risk event are extracted to determine the multimodal data source within the corresponding spatiotemporal range; Centered on the triggering time of a risk event, and based on a preset time window corresponding to the risk type, raw data within the time window range is obtained from various multimodal data sources. Feature extraction and evidence anchor point generation are performed on the raw data, which are mapped into standardized evidence units and summarized to form the evidence unit set. Perform cross-modal consistency verification on the evidence units in the evidence unit set, determine the type of association relationship between each pair of evidence units, and construct the association logic set; A comprehensive confidence score is calculated based on the completeness of the evidence unit set and the consistency of the conclusions of the related logic set, and mapped to the corresponding confidence level. If the confidence level is to be supplemented, a multi-round dynamic expansion verification mechanism is triggered to expand the search scope to obtain supplementary evidence to update the evidence unit set and the related logic set, and the confidence level is recalculated until the confidence level is no longer to be supplemented or the maximum number of iterations is reached, and the final multimodal risk evidence chain is output.

2. The intelligent collection method for multimodal risk evidence chains for safety supervision of transportation engineering construction according to claim 1, characterized in that: Each evidence unit is a five-tuple structure, containing unit identifier, evidence type, evidence content, evidence anchor information, and collection timestamp; the evidence types include on-site images, IoT monitoring, inspection logs, engineering documents, standard clauses, business data, and scenario context; the relationship types include supporting relationships, supplementary relationships, and conflicting relationships; The output multimodal risk evidence chain meets the feasible evidence chain constraints: the total number of evidence units is not less than 3, all evidence units contain non-empty evidence anchor information, and the required evidence slot table corresponding to the risk type covers the corresponding on-site facts, business or monitoring data, specifications or engineering basis; scenarios without a certain type of physical data source are constrained by alternative evidence slots and manual review requirements.

3. The intelligent collection method for multimodal risk evidence chains for safety supervision of transportation engineering construction according to claim 2, characterized in that: The cross-modal consistency verification adopts corresponding verification rules based on the combination of evidence types, specifically including: For the combined evidence of on-site video and IoT monitoring, the visual anomaly degree of the video and the anomaly degree of the IoT monitoring are calculated separately. A relationship is considered supportive if the absolute value of the difference between the two is not greater than a first preset threshold; otherwise, a conflicting relationship is considered. For upper limit indicators, the IoT monitoring anomaly degree is calculated based on the indicator's risk direction, warning value, alarm value, and rate of change. The calculation formula is as follows: In the formula, These are actual measured values ​​from IoT sensors. The indicator's warning threshold, The indicator alarm threshold, This is a quantitative value representing the degree of change in the indicator. For the combination of regulatory provisions and evidence from on-site images or IoT monitoring, the regulatory provisions are parsed into a set of constraints containing predicates, objects, and limits. The on-site evidence is converted into a set of feature predicates. First, a risk proposition H to be verified is defined. If the constraint set matches the semantics of the on-site features and the on-site values ​​violate the regulatory limits, then the regulatory evidence supports the risk proposition H. If the on-site values ​​meet the regulatory limits, then a rebuttal relationship is formed. No association is established when the provisions are inapplicable or semantically mismatched. For the combination of engineering documents and on-site images or IoT monitoring evidence, verify the consistency of the work sites and monitoring indicators between the engineering documents and on-site evidence. If they are consistent, they are determined to be complementary; otherwise, no association is established.

4. The intelligent collection method for multimodal risk evidence chains for safety supervision of transportation engineering construction according to claim 1, characterized in that: The preset time windows are set according to risk type, with a window Tpre before the event occurs and a window Tpost after the event occurs, and corresponding scenario-based adjustment rules are set; "±Th" means Tpre=Tpost=Th: The basic time window for deep foundation pit risk is ±24h. If the time consistency deviation is greater than the basic time window and the IoT sampling frequency is not less than 1 time / h, the time window will be extended by 12h, with a maximum of 48h. The basic time window for tunnel risks is ±12h, the time window for gas-related risks is shortened to ±6h, and the time window for abnormal ventilation scenarios is fixed at ±3h. The basic time window for scaffolding risks is ±8 hours. If only images are triggered and there is no IoT data, the time window will be increased by 4 hours based on the baseline value. The basic time window for high slope risk is ±24h. When the daily rainfall is greater than 25mm / d, the time window is extended to ±36h. The basic time window for edge protection risks is ±4 hours, and the time window for events triggered by manual reporting is shortened to ±2 hours. The time window is also dynamically adjusted using a general adaptive rule, and the adjusted time window satisfies the following relationship: In the formula, As the base time window for the current round, This is the maximum upper limit of the time window. This represents the number of key evidence types missing during the initial alignment.

5. The intelligent collection method for multimodal risk evidence chains for safety supervision of transportation engineering construction according to claim 1, characterized in that: The maximum number of iterations for the multi-round dynamic extended verification mechanism is 3. Each expansion simultaneously performs spatial breadth expansion and temporal depth expansion. Spatially, it prioritizes expansion along adjacent nodes of WBS, the same structural unit, the same monitoring section, and the camera field of view coverage relationship. If necessary, it expands the search range according to the configurable physical radius. Temporally, it expands the temporal sliding window increment. After updating the spatiotemporal boundaries, the evidence extraction, association construction, and confidence assessment processes are re-executed.

6. The intelligent collection method for multimodal risk evidence chains for safety supervision of transportation engineering construction according to claim 1, characterized in that: The overall confidence score is calculated by weighting the evidence completeness score and the conclusion consistency score; the evidence completeness score is determined based on the sub-scores of five dimensions: image evidence, IoT evidence, normative matching, anchor points, and logical connections; and the conclusion consistency score is the average of all quantifiable evidence against the consistency measure. Based on the comprehensive confidence score, the confidence level is mapped to four levels: high confidence, medium confidence, low confidence, and pending further evidence. The contribution rules of the relationship to the confidence score are as follows: supporting relationship participates in the positive calculation of the conclusion consistency score, supplementary relationship improves the evidence completeness score but does not participate in conflict detection, conflict relationship triggers IoT data re-collection or time window expansion for supplementary evidence, and if there is still a conflict, it will be transferred to manual review and the confidence level will be reduced by one level. When the confidence level is "to be supplemented", a supplementary verification task is generated and dispatched to the retrieval module or on-site inspection terminal. If the supplementary verification task is not completed within the deadline, it is escalated to the security administrator. If it expires twice, the confidence level of the evidence chain will be forcibly set to "to be supplemented" and the automatic handling process of the risk event will be blocked.

7. The intelligent collection method for multimodal risk evidence chains for safety supervision of transportation engineering construction according to claim 1, characterized in that: The evidence anchor information includes the source system identifier, collection timestamp, file number, standard clause number, document page number, and data hash value; the structured data hash value is generated by calculating the normalized JSON content after sorting and removing whitespace using the SHA-256 algorithm; the hash of image, video, and other file-type evidence is calculated by combining the original file binary, source handle, and key metadata. Multimodal data sources are encapsulated into a unified calling interface through model context protocol or function call specification, and data is obtained by intelligent agents in a low-intrusive manner and uniformly mapped into evidence units in standardized JSON format; When performing evidence structuring, engineering structure parameters, historical risk records, and equipment location attribute spatiotemporal subgraph data corresponding to the spatiotemporal range are extracted from the dynamic configurable knowledge graph based on the spatiotemporal anchor points of risk events, and classified into scenario context-based evidence units to supplement relationships and incorporate them into the evidence chain.

8. The intelligent collection method for multimodal risk evidence chains for safety supervision of transportation engineering construction according to claim 1, characterized in that: The feature extraction methods for raw data of different modalities are as follows: Image data uses object detection and scene classification models to extract features, and the output includes the object, label and severity score. For IoT data, a sliding window method is used to calculate the mean, slope, or number of out-of-bounds errors. For text-based data, risk entities and constraint predicates are extracted using named entity recognition.

9. A multimodal risk evidence chain intelligent acquisition system for safety supervision of transportation engineering construction, characterized in that, include: The anchor configuration module is used to construct a quadruple representation model of a multimodal risk evidence chain and establish a spatial, temporal, and semantic triple anchor alignment benchmark. The spatial anchor points are located using a combination of work breakdown structure coding and geographic coordinates or measurement point numbers. The event triggering module is used to receive risk event triggering information, extract the corresponding spatiotemporal anchor point, and determine the multimodal data source within the corresponding spatiotemporal range; The evidence extraction module is used to obtain raw data from various multimodal data sources based on a preset time window, with the trigger time of the risk event as the center, generate standardized evidence units, and summarize them to form an evidence unit set. The association construction module is used to perform cross-modal consistency verification on evidence units, determine the type of association relationship between evidence units, and construct an association logic set. The confidence assessment module is used to calculate a comprehensive confidence score and map a confidence level based on the completeness of evidence and the consistency of conclusions. If the confidence level is to be supplemented, a multi-round dynamic expansion verification mechanism is triggered to update the evidence chain until the confidence level is no longer to be supplemented or the maximum number of iterations is reached, and the final multimodal risk evidence chain is output.

10. The intelligent acquisition system for multimodal risk evidence chains for safety supervision of transportation engineering construction according to claim 9, characterized in that, It also includes a dynamic expansion and supplementary certificate management module, which is used to trigger a multi-round dynamic expansion verification mechanism when the confidence level is pending supplementary certificate, generate and dispatch supplementary certificate tasks, and manage the supplementary certificate timeout upgrade process and automatic handling blocking logic.

Images