Construction process safety management multi-modal large model construction method and system

Abstract

This application provides a method and system for constructing a multimodal large-scale model for safety management in the construction process. The method includes: acquiring structured and unstructured data related to safety management in the construction process, constructing a safety element ontology, and assigning global object identifiers and relational constraints to objects in the structured and unstructured data; processing the structured and unstructured data, generating evidence objects based on global object identifiers and unified time stamps; constructing six categories of libraries based on the evidence objects, using the evidence objects and entries associated with object identifiers in the six categories as training samples, performing multimodal representation alignment training and consistency verification training based on relational constraints to obtain a multimodal large-scale model for safety management in the construction process, and deploying the multimodal large-scale model as a reasoning model for risk classification, hazard closure-loop judgment, and emergency resource allocation based on evidence object references. This application can improve the accuracy of risk identification, accelerate hazard closure-loop handling, and improve emergency response efficiency.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a method and system for constructing a multimodal large model for safety management of building construction processes. Background Technology

[0002] Construction sites involve diverse work types, frequent personnel movement, and dynamic changes in equipment, facilities, and environmental conditions. Safety management requires continuous updating and coordinated handling of information related to objectives and responsibilities, site management, equipment and facility management, environmental and occupational health management, risk classification, hazard identification and mitigation, and emergency and accident management. Current technologies typically rely on ledger systems, checklists, and video surveillance for safety management. Structured data is mostly manually entered to form risk assessment forms, hazard rectification orders, emergency plans, and accident reports. Unstructured data primarily includes on-site videos, inspection photos, and sensor data, which are often stored separately in different systems, lacking a unified data caliber and semantic framework.

[0003] The lack of unified identification and relational constraints for personnel, areas, tasks, equipment, and hazards makes it difficult to achieve cross-source object alignment and spatiotemporal correlation, resulting in delayed risk classification updates, incomplete evidence of hazard closed-loop links, disconnect between emergency resource allocation and on-site situation, and difficulty in forming reusable structured knowledge from accident debriefing. At the same time, multimodal data has failed to form searchable evidence objects and training samples, and safety analysis relies on human experience, making it difficult to achieve continuous and auditable intelligent management under changes in the construction process. Summary of the Invention

[0004] In view of this, the embodiments of this application provide a method and system for constructing a multimodal large model for safety management in the construction process, in order to solve the problems of existing technologies, such as difficulty in semantic unification and object alignment of multi-source data, difficulty in dynamic and reliable closed-loop management of risks and hazards, and difficulty in evidence citation and consistency verification for emergency decisions.

[0005] A first aspect of this application provides a method for constructing a multimodal large model for safety management in building construction processes, comprising: acquiring structured and unstructured data of safety management in building construction processes, and constructing a safety element ontology; assigning global object identifiers and relational constraints to objects in the structured and unstructured data based on the safety element ontology; performing field normalization, semantic mapping, and spatiotemporal alignment processing on the structured and unstructured data; generating evidence objects based on global object identifiers and unified time stamps; and constructing six types of libraries based on the evidence objects, including a personnel information library, a knowledge library, a risk management library, a hazard investigation library, an emergency management library, and an accident management library. The personnel information library stores job qualifications and authorization scope associated with personnel object identifiers; the knowledge library stores the association between clause objects and job types and risk types; the risk management library stores dynamic risk entries and classification criteria associated with risk source object identifiers; and the hazard investigation library stores closed-loop events associated with hazard object identifiers. The emergency management database stores the plan versions and resource object availability status associated with scenario objects, while the accident management database stores the evidence chain and cause chain associated with accident object identifiers. Using evidence objects and entries associated with object identifiers in the six databases as training samples, multimodal representation alignment training and consistency verification training based on relational constraints are performed to obtain a large multimodal model for safety management of the construction process. The large multimodal model is then deployed as a reasoning model for risk classification, hazard closed-loop judgment, and emergency resource allocation based on evidence object references.

[0006] A second aspect of this application provides a multimodal large-scale model construction system for safety management in building construction processes, comprising: an acquisition module for acquiring structured and unstructured data related to safety management in building construction processes, constructing a safety element ontology, and assigning global object identifiers and relational constraints to objects in the structured and unstructured data based on the safety element ontology; a generation module for performing field normalization, semantic mapping, and spatiotemporal alignment processing on the structured and unstructured data, and generating evidence objects based on global object identifiers and unified time stamps; and a construction module for constructing six types of libraries based on the evidence objects, including a personnel information library, a knowledge library, a risk management library, a hazard investigation library, an emergency management library, and an accident management library. The personnel information library stores job qualifications and authorization scope associated with personnel object identifiers; the knowledge library stores the association between clause objects and job types and risk types; the risk management library stores dynamic risk entries and classification criteria associated with risk source object identifiers; and the hazard investigation library stores closed-loop events associated with hazard object identifiers. The emergency management database stores the versions of contingency plans and the availability status of resource objects associated with scenario objects, while the accident management database stores the evidence chains and causal chains associated with accident object identifiers. The training module uses evidence objects and entries associated with object identifiers in the six databases as training samples to perform multimodal representation alignment training and consistency verification training based on relational constraints, thereby obtaining a large multimodal model for safety management of the construction process. The large multimodal model is then deployed as a reasoning model based on risk classification, hazard closed-loop judgment, and emergency resource allocation based on evidence object references.

[0007] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the steps of the above-described method.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects: By acquiring structured and unstructured data on safety management during the construction process and constructing a safety element ontology, global object identifiers and relational constraints are assigned to objects in both structured and unstructured data based on the safety element ontology. Field normalization, semantic mapping, and spatiotemporal alignment are performed on the structured and unstructured data, and evidence objects are generated based on global object identifiers and unified time stamps. Six types of databases are constructed based on these evidence objects: a personnel information database, a knowledge base, a risk management database, a hazard identification database, an emergency management database, and an accident management database. Specifically, the personnel information database stores job qualifications and authorization scope associated with personnel object identifiers; the knowledge base stores the association between clause objects and job types and risk types; the risk management database stores dynamic risk entries and classification criteria associated with risk source object identifiers; and the hazard identification database stores closed-loop events associated with hazard object identifiers. The emergency management database stores the versions of contingency plans and the availability status of resource objects associated with scenario objects, while the accident management database stores the evidence chains and causal chains associated with accident object identifiers. Using evidence objects and entries associated with object identifiers from six categories of databases as training samples, multimodal representation alignment training and consistency verification training based on relational constraints are performed to obtain a large-scale multimodal model for safety management in the construction process. This large-scale multimodal model is then deployed as a reasoning model based on evidence object references for risk classification, hazard closure-loop judgment, and emergency resource allocation. This application can improve the accuracy of risk identification, accelerate hazard closure-loop handling, and enhance emergency response efficiency. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart illustrating the method for constructing a multimodal large model for safety management of building construction processes provided in this application embodiment; Figure 2 This is a schematic diagram of the structure of the multimodal large-scale model construction system for safety management of building construction process provided in the embodiments of this application; Figure 3 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0012] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0013] In existing technologies, safety management during building construction typically relies on methods such as ledger systems, checklists, and video surveillance to conduct risk classification, hazard identification and mitigation, and emergency and accident management. Structured data is mostly manually entered and stored in a scattered manner, while unstructured data mainly consists of videos, inspection images, and sensor time series. The two lack a unified standard and semantic system, making it difficult to align objects across systems and form searchable, auditable chains of evidence and training samples. This results in safety analysis and handling relying heavily on human experience.

[0014] Based on the aforementioned existing technologies, the technical problems to be solved by this application are: multi-source heterogeneous data are difficult to unify semantics and align with objects; risk classification and hidden danger management are difficult to be dynamically updated with the construction process and form a credible closed loop; emergency resource allocation is difficult to be consistent with the on-site situation and evidence citation; and accident review is difficult to be precipitated into reusable structured knowledge.

[0015] To address the aforementioned technical issues, this application proposes a method for constructing a multimodal large-scale model for safety management in building construction processes: A safety element ontology covering personnel, positions, tasks, regional grids, equipment and facilities, and hazard sources is constructed. Global object identifiers are assigned to objects in multi-source data, and relational constraints are configured. Field normalization, semantic mapping, and spatiotemporal alignment are performed on structured and unstructured data. Evidence objects are encapsulated based on global object identifiers and unified time stamps. Furthermore, personnel information databases, knowledge bases, risk management databases, hazard investigation databases, emergency management databases, and accident management databases are constructed based on the evidence objects. Training samples are formed using the evidence objects and database entries. Multimodal representation alignment training and consistency verification training based on relational constraints are performed to obtain a multimodal large-scale model capable of outputting evidence references and relational references. This model is then deployed as a reasoning model for risk classification, hazard closed-loop judgment, and emergency resource allocation.

[0016] By adopting the above technical solutions, this application can improve the accuracy and consistency of multi-source data fusion and object association, enhance the accuracy and real-time performance of risk identification and classification, accelerate the closed-loop handling of hidden dangers and the traceability of evidence, improve the response efficiency and decision reliability of emergency resource allocation, and enhance the knowledge accumulation and cross-project reuse capabilities of accident review.

[0017] The technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0018] Figure 1 This is a flowchart illustrating the method for constructing a multimodal large model for safety management of the building construction process provided in this application. For example... Figure 1 As shown, the method for constructing a multimodal large model for safety management during building construction can specifically include: S101: Obtain structured and unstructured data for safety management during construction, construct a safety element ontology, and assign global object identifiers and relational constraints to objects in the structured and unstructured data based on the safety element ontology. S102, perform field normalization, semantic mapping and spatiotemporal alignment processing on structured and unstructured data, and generate evidence objects based on global object identifiers and unified time stamps; S103, based on evidence objects, construct six types of databases, including personnel information database, knowledge base, risk management database, hidden danger investigation database, emergency management database, and accident management database. Among them, the personnel information database stores the job qualifications and authorization scope associated with personnel object identifiers; the knowledge base stores the relationship between clause objects and operation types and risk types; the risk management database stores dynamic risk items and classification basis associated with risk source object identifiers; the hidden danger investigation database stores closed-loop chain events associated with hidden danger object identifiers; the emergency management database stores the plan version and resource object availability status associated with scenario objects; and the accident management database stores the evidence chain and cause chain associated with accident object identifiers. S104 uses the evidence object and the entries associated with the object identifier in the six categories of the library as training samples to perform multimodal representation alignment training and consistency verification training based on relational constraints, thereby obtaining a multimodal large model for safety management of the construction process. The multimodal large model is then deployed as a reasoning model based on risk classification, hazard closed-loop judgment, and emergency resource allocation based on evidence object references.

[0019] In some embodiments, structured and unstructured data on safety management during the construction process are acquired, and a safety element ontology is constructed, including: Acquire structured and unstructured data for safety management during the construction process, and record data source identifiers and collection time markers for structured and unstructured data respectively; among them, structured data includes target and responsibility information, on-site management information, equipment and facility information, environmental information, occupational health information, risk classification information, hidden danger investigation and management information, emergency management information, accident management information and inspection records, and unstructured data includes on-site video data, sensor time series data and risk map data; The safety element ontology is constructed based on a pre-defined set of safety objects and a set of relationship types. Safety objects include personnel, positions, work tasks, regional grids, equipment and facilities, hazard sources, risk levels, control measures, hidden dangers, emergency resources and accident events. Relationship types include subordinate relationships, association relationships, spatiotemporal inclusion relationships and handling link relationships. Relationship constraints are configured for the safety element ontology. For data items in both structured and unstructured data, semantic mapping and object normalization based on security element ontology are performed to map personnel identifiers, equipment identifiers, area identifiers, or task identifiers in data items to object identifiers corresponding to security objects.

[0020] Specifically, the safety element ontology refers to a semantic model used to characterize the object categories, object attributes, types of relationships between objects, and their constraint rules in the field of construction safety. It can abstract personnel, positions, work tasks, regional grids, equipment and facilities, hazard sources, risk levels, control measures, hidden dangers, emergency resources, and accident events into computable safety objects, and define the subordinate relationships, association relationships, spatiotemporal inclusion relationships, and handling link relationships between safety objects.

[0021] Object normalization refers to the process of merging and unifying the multiple identifiers of the same real-world object from different data sources, so that subsequent processing relies only on the unified object identifier and no longer depends on the local identifiers on the source side. Relationship constraints refer to the rules that constrain the types of relationships that can be established between secure objects, the cardinality of relationships, and the effective time range of relationships, used to ensure consistency during data writing and association.

[0022] In some examples, structured and unstructured data of safety management during the construction process are acquired, and data source identifiers and collection time stamps are recorded for the structured and unstructured data respectively. In this embodiment, structured data may include target and responsibility information, on-site management information, equipment and facility information, environmental information, occupational health information, risk classification information, hazard identification and mitigation information, emergency management information, accident management information, and inspection records.

[0023] For example, target and responsibility information includes project safety objectives, job responsibility lists, and responsible personnel lists; on-site management information includes work permit records, pre-shift meeting records, and work process plans; equipment and facility information includes equipment ledgers and maintenance records for tower cranes, elevators, distribution boxes, etc.; environmental information includes monitoring records for meteorology, dust, noise, etc.; occupational health information includes personnel medical examination summaries and protective equipment distribution records; risk classification information includes a list of hazard sources, risk levels, and control measures; hazard investigation and management information includes a closed-loop record of hazard discovery, dispatch, rectification, and re-inspection; emergency management information includes emergency plan versions, emergency resource lists, and drill records; accident management information includes accident reports, cause analysis, and rectification verification records; and inspection records include checklist items, inspection conclusions, and rectification suggestions. Unstructured data in this embodiment may include on-site video data, sensor time-series data, and risk map data. On-site video data comes from fixed cameras or mobile law enforcement recorders; sensor time-series data comes from dust sensors, combustible gas sensors, temperature and humidity sensors, equipment vibration sensors, or personnel positioning tags; and risk map data is risk distribution information divided into regional grids on the construction site plan.

[0024] To ensure data traceability, this embodiment establishes a data source identifier and a collection time stamp for each structured record and each segment of unstructured data. For example, the safety management platform interface is identified as SRC01, the tower crane monitoring gateway is identified as SRC02, and the video camera location is identified as CAM03. The collection time stamp is uniformly recorded in the format YYYYMMDDhhmmss or an equivalent unified time expression.

[0025] Furthermore, a safety element ontology is constructed based on a preset set of safety objects and a set of relationship types, and relational constraints are configured for the safety element ontology. In this embodiment, the set of safety objects includes, but is not limited to: personnel, positions, work tasks, regional grids, equipment and facilities, hazard sources, risk levels, control measures, hidden dangers, emergency resources, and accident events. The set of relationship types includes hierarchical relationships, association relationships, spatiotemporal inclusion relationships, and handling link relationships.

[0026] For example, affiliation is used to establish the affiliation of personnel with positions, personnel with work groups, and equipment with projects; association is used to establish the association between work tasks and hazards, hidden dangers and hazards, and control measures and risk levels; spatiotemporal inclusion is used to establish the inclusion of personnel locations with regional grids, equipment layouts with regional grids, and work task occurrence areas with regional grids; and handling link is used to establish the event link from the discovery of hidden dangers to re-verification, the event link from the occurrence of accidents to rectification and verification, and the link between emergency events and resource allocation actions.

[0027] In this embodiment, relational constraints are used to limit the object category pairs and cardinality of relational connections. For example, they limit the relationship between personnel and positions to a single subordinate relationship, allow many-to-many associations between work tasks and hazards, require the hazard handling chain to include discovery events and re-inspection events with the time sequence satisfying the increasing collection time stamp, and require emergency resource allocation actions to be associated with emergency resource objects in an available state. This allows for constraint verification during subsequent data writing and association.

[0028] Furthermore, after completing the construction of the safety element ontology, semantic mapping and object normalization based on the safety element ontology are performed on data items in both structured and unstructured data to map personnel identifiers, equipment identifiers, area identifiers, or task identifiers in the data items to object identifiers corresponding to the safety objects. In this embodiment, semantic mapping is used to align fields from different sources with ontology attributes. For example, the worker_id field in the safety management platform is mapped to a personnel identifier attribute, the asset_no field in the equipment ledger is mapped to a device identifier attribute, the permit_id field in the work permit is mapped to a work task identifier attribute, and the grid_code field in the risk map is mapped to a region grid identifier attribute; at the same time, synonymous fields such as equipment number, asset number, and QR code number are mapped to the same equipment identifier attribute.

[0029] Object normalization is used to merge multiple source identifiers of the same object. For example, a tower crane appears as asset number TC-001 in the equipment ledger, as equipment serial number SN-9A7B in the monitoring gateway, and as QR code number QR-TC001 in the inspection record. In this embodiment, based on a preset normalization rule, the above identifiers are aggregated into the same equipment facility object, and a unified object identifier is generated for subsequent reference. As another example, a work area is represented by grid G12 in the plan view and by area code AREA-12 in the video point configuration. It is then normalized into the same area grid object and a unified object identifier is generated.

[0030] For personnel objects, this embodiment can be unified based on employee ID, real-name information, and location tag number, associating personnel with job qualification records, training records, and on-site location sequence to the same personnel object identifier. Through the above semantic mapping and object unification processing, key objects in structured and unstructured data are uniformly mapped to object references under the security element ontology, thereby enabling subsequent steps to perform cross-source association and consistency constraints based on object identifiers.

[0031] The following is a specific example: During the parallel construction of high-altitude operations and hoisting in a building construction project, the safety management platform generates a work permit record, recording the work task identifier PERMIT-20260128-001, the responsible person Zhang San's work number A102, the work area G12, and the hazard source type hoisting; the video camera CAM03 captures the hoisting operation footage during the same time period; the tower crane monitoring gateway SRC02 collects the time-series data of the tower crane's amplitude and load; and the risk map system provides the risk level and control measures corresponding to area G12 during that time period.

[0032] According to this embodiment, the system records the data source identifier and collection time stamp for the above data records respectively, and under the safety element ontology, A102 is unified as a personnel object identifier, the tower crane multi-source identifier is unified as an equipment and facility object identifier, G12 is unified as a regional grid object identifier, and PERMIT-20260128-001 is mapped as a work task object identifier; at the same time, based on relational constraints, the system establishes the personnel and job affiliation relationship, the work task and regional grid spatiotemporal inclusion relationship, the work task and hazard source association relationship, and the hazard source and risk level association relationship, so that subsequent processing can continue to generate evidence objects and carry out training sample construction under the same object identifier system.

[0033] The processing method in this embodiment can form a unified data source identifier and collection time mark during the data access stage, and realize multi-source object semantic mapping and object unification with the support of security element ontology and relation constraints. This reduces the probability of object mismatch caused by inconsistent data standards across systems, improves the availability of subsequent evidence object generation and cross-modal alignment, and provides a consistent object reference basis and relation constraint basis for model training related to risk classification, hidden danger closure and emergency deployment.

[0034] In some embodiments, global object identifiers and relational constraints are assigned to objects in structured and unstructured data based on the security element ontology, including: Configure object identification rules for various security objects in the security element ontology. The object identification rules include object category code and field scope code, and generate object fingerprints based on the key attributes of the object to obtain the global object identifier corresponding to the object. For the same object with multiple identification information from different data sources, cross-source deduplication and merging are performed based on the object fingerprint to determine the master object record, and the source identifiers associated with the master object record are written into the object identifier mapping table; A set of relation constraint rules is generated based on the set of relation types of the security element ontology. The set of relation constraint rules includes the object category pairs that the relation can connect to, the cardinality constraint of the relation, and the effective time range constraint of the relation. The global object identifier, object identifier mapping table, and relation constraint rule set are associated and stored. When writing index records of structured or unstructured data, the objects are uniformly referenced as global object identifiers based on the object identifier mapping table, and constraint verification is performed on the relationships between objects based on the relation constraint rule set.

[0035] Specifically, the object category code refers to the coded information used to identify the category to which a safety object belongs. Different categories correspond to different sets of safety objects. For example, personnel, equipment and facilities, area grids, and work tasks each correspond to different object category codes. The site scope code refers to the coded information used to characterize the project, section, or work area to which the object belongs. The site scope code is used to avoid object identification conflicts across projects or sections.

[0036] An object fingerprint refers to a feature summary generated based on the key attributes of an object, which can be used for cross-source comparison. The key attributes of an object vary depending on the object category. For example, the key attributes of a personnel object may include a real name identity summary, an employee ID summary, and a location terminal number summary. The key attributes of an equipment and facility object may include an asset number summary, an equipment serial number summary, and a QR code number summary. The key attributes of a regional grid object may include a grid code summary and a planar coordinate summary.

[0037] The object identifier mapping table is a mapping data structure used to record the correspondence between multi-source identifiers and global object identifiers. It is used to uniformly replace the source-side identifiers with global object identifiers when writing data. The relationship constraint rule set refers to the set of constraint rules configured for relationship types in the security element ontology. It is used to limit at least the object category pairs that the relationship can connect, the relationship cardinality, and the effective time range of the relationship, so as to perform constraint verification when writing object relationships and performing association calculations.

[0038] In some examples, object identification rules are configured for various safety objects within the safety element ontology. These rules include object category codes and site scope codes, and object fingerprints are generated based on the object's key attributes to obtain a global object identifier corresponding to the object. In this embodiment, the system determines an object category code for each type of safety object; for example, personnel are set to 01, positions to 02, work tasks to 03, area grids to 04, equipment and facilities to 05, hazard sources to 06, hidden dangers to 07, emergency resources to 08, and accidents / events to 09. The site scope code is used to distinguish between projects and sections. For example, if a project is coded as P2026 and a section is coded as S1, the site scope code can be combined as P2026S1.

[0039] The generation of object fingerprints employs a category-adaptive key attribute combination strategy: for personnel objects, it selects a summary of real name information, employee ID, unit or work group affiliation, and location tag number; for equipment and facility objects, it selects an asset number, equipment serial number, equipment model summary, and installation point summary; for area grid objects, it selects a grid code and a plan coordinate range summary; for work task objects, it selects a work permit number, work type summary, work area grid identifier, and planned start and end time summary. The system combines the object category code, site range code, and object fingerprint to generate a global object identifier, enabling the same object to form a stable and reusable unified identifier within the same site area, while simultaneously isolating different site areas.

[0040] Furthermore, for multiple identifiers of the same object from different data sources, cross-source deduplication and merging are performed based on object fingerprints to determine the master object record. The identifiers from each source associated with the master object record are then written into the object identifier mapping table. In this embodiment, cross-source deduplication and merging is based on object fingerprints, combined with source credibility for master record selection. For example, a personnel object may appear as an ID card summary and employee number in the real-name system, as an employee badge number in the access control and attendance system, and as a location tag number in the location system. The system merges these multi-source identifiers into the same personnel object through object fingerprint comparison, prioritizing the record from the real-name system as the master object record, and writing the employee badge number, location tag number, and attendance system internal number into the object identifier mapping table to form a mapping from multi-source identifiers to global object identifiers.

[0041] Equipment and facility objects may appear as asset numbers in the equipment ledger, as equipment serial numbers in the monitoring gateway, and as QR code numbers in the inspection record. The system merges multi-source identifiers based on object fingerprint comparison, using the asset number record in the equipment ledger as the master object record, and mapping the serial number and QR code number to the same global object identifier. For regional grid objects, the grid codes in the plan view, the regional codes in the risk map, and the regional codes in the video point configuration often have different naming methods. The system merges them based on the grid coordinate range summary and coding mapping rules, and records the corresponding relationship of each code in the object identifier mapping table. Through the object identifier mapping table, the source-side identifier can be directly replaced with the global object identifier in the subsequent data writing stage, avoiding the same object being repeatedly modeled as multiple object entries.

[0042] Furthermore, a set of relationship constraint rules is generated based on the set of relationship types of the security element ontology. This set includes object category pairs that can be connected by the relationship, relationship cardinality constraints, and relationship validity time range constraints. In this embodiment, the system generates constraint rules for subordinate relationships, association relationships, spatiotemporal inclusion relationships, and handling link relationships, respectively.

[0043] In some examples, taking affiliation as an example, it restricts personnel to only one position or allows only one primary position affiliation within the same valid time frame; a position can belong to a work group or unit. Taking association as an example, it restricts work tasks to be associated with multiple hazards, and each hazard can be associated with multiple control measures and risk level entries. Taking spatiotemporal inclusion as an example, it restricts personnel location records to be associated with a regional grid, and the valid time frame of this relationship must be consistent with the data collection time stamp; the area where the work task occurs must be contained within a pre-configured set of regional grids. Taking hazard handling chain relationships as an example, it restricts hazard handling chains to include at least a discovery event and a re-verification event, and the data collection time stamp of the re-verification event must not be earlier than the discovery event; in the accident event chain, the cause analysis event should be after the accident occurrence event and before the rectification verification event. Through these relationship constraints, the system can directly determine whether the relationship type is allowed, whether the number exceeds the limit, and whether the time frame conflicts when forming relationships between objects.

[0044] Furthermore, the global object identifier, object identifier mapping table, and relationship constraint rule set are associated and stored together. When writing index records of structured or unstructured data, objects are uniformly referenced as global object identifiers based on the object identifier mapping table, and constraint verification is performed on the relationships between objects based on the relationship constraint rule set. In this embodiment, the associated storage includes binding the global object identifier to its object fingerprint, master object record, and source identifier list; binding the relationship constraint rule set to the security element ontology version; and recording the effective time range for the object identifier mapping table to support subsequent version evolution and backtracking verification.

[0045] During the writing phase, for each structured record or unstructured index record, the system first reads the personnel identifier, equipment identifier, area identifier, and task identifier. It then retrieves the corresponding global object identifier through the object identifier mapping table and replaces it. Next, it verifies whether the object relationship to be established for the record satisfies the constraints based on the set of relationship constraint rules. For example, when an inspection record attempts to assign the same person to two positions under the same time marker, the system determines a conflict based on the cardinality constraint of the affiliation relationship and marks it as pending review. When a hazard record lacks a review event but is written as closed-loop, the system determines the link is incomplete based on the handling link relationship constraint and prevents state transition. When the area code of a video index record cannot be mapped to any configured area grid object, the system determines that the spatiotemporal inclusion relationship cannot be established and triggers the area mapping completion process.

[0046] The following is a specific example: In project P2026S1, the tower crane has asset number TC-001 in the equipment ledger, equipment serial number SN-9A7B in the monitoring gateway, and QR code number QR-TC001 in the inspection record. The system sets the equipment and facility object category code to 05 and the site range code to P2026S1, extracts the asset number, serial number, and QR code number to generate an object fingerprint, and obtains a global object identifier for the same object; then, TC-001, SN-9A7B, and QR-TC001 are written into the object identifier mapping table and associated with this global object identifier. The work permit record references the equipment identifier TC-001 and the area grid G12, the video index record references the equipment serial number SN-9A7B and the area code AREA-12, and the sensor time sequence record references the QR code number QR-TC001.

[0047] Furthermore, during the writing phase, the system uniformly replaces TC-001, SN-9A7B, and QR-TC001 with the same global object identifier, and unifies G12 and AREA-12 to the same regional grid object identifier. At the same time, based on the relational constraints, the system verifies that the effective time range of the spatiotemporal inclusion relationship between the task and the regional grid is consistent with the acquisition time marker, and verifies that the layout relationship between the equipment and facilities and the regional grid has a unique mapping within the field area, thereby ensuring that subsequent cross-modal evidence can be aggregated and aligned around the same global object identifier and the same regional grid object identifier.

[0048] The processing method in this embodiment can establish stable global object identifiers for key objects in multi-source heterogeneous data, and realize unified cross-source identifier referencing through an object identifier mapping table. At the same time, it uses a set of relation constraint rules to perform constraint verification during the data writing and relation establishment stages, reducing data inconsistencies caused by repeated object modeling and relation conflicts. This provides a reliable object reference foundation and relation consistency foundation for subsequent evidence object generation, six types of library entries construction, and consistency verification training, thereby improving the usability and credibility of building construction process safety management data fusion and subsequent intelligent reasoning.

[0049] In some embodiments, field normalization, semantic mapping, and spatiotemporal alignment are performed on structured and unstructured data, and evidence objects are generated based on global object identifiers and unified time stamps, including: The metadata repository reads the field definition rules and enumeration mapping rules associated with each data source identifier. Based on the field definition rules, the structured data is aligned, units are standardized, and missing values ​​are handled. Based on the enumeration mapping rules, the field values ​​are semantically normalized to generate standardized structured records. Extract the collection point identifier and collection time information from unstructured data, and determine the spatial location mark based on the mapping relationship between the collection point identifier and the regional grid; perform frame-level timestamp calibration on the on-site video data, perform sampling time recalibration on the sensor time series data, and perform grid coordinate registration on the risk map data to generate multimodal index records aligned with a unified time reference; Based on the object identifier mapping table, object references in standardized structured records and multimodal index records are unified into global object identifiers, and a unified time stamp associated with the global object identifier is generated using a unified time base. Evidence objects are generated according to a preset evidence encapsulation structure. The evidence objects include content payload, data source identifier, unified time stamp, spatial location marker, associated global object identifier, and integrity summary.

[0050] Specifically, a metadata repository refers to a storage space used to centrally store governance metadata such as data source identifiers, field definition rules, enumeration mapping rules, collection point configurations, regional grid mapping relationships, and time base configurations.

[0051] Field definition rules refer to a set of rules governing the names, data types, units, value ranges, and mandatory constraints of structured data fields. These rules are used to align fields across different business systems with consistent semantics. Enumeration mapping rules refer to a set of rules for uniformly encoding and semantically aligning enumeration values, status values, and type values ​​in structured data. For example, mapping "rectified," "rectification completed," and "processed" from different systems to the same status code.

[0052] Sampling time recalibration refers to remapping or interpolating the sampling times of sensor time-series data to align them with a unified time reference and satisfy sampling interval consistency constraints. Grid coordinate registration refers to the process of mapping the grid coordinate system in the risk map to the grid coordinate system of the construction site area.

[0053] Integrity summary refers to summary information generated based on the content payload of the evidence object, and is bound to the data source identifier, unified time stamp and associated global object identifier, for subsequent consistency verification and evidence citation verification.

[0054] A unified time base refers to an internal reference time system used to unify the time of different data sources. It can be based on the server time source or the project time service, and compensates for the clock deviation of each data source.

[0055] In some examples, the system reads field definition rules and enumeration mapping rules associated with each data source identifier from the metadata repository. Based on the field definition rules, it performs field alignment, unit unification, and missing value handling on the structured data. It also performs semantic normalization on the field values ​​based on the enumeration mapping rules to generate standardized structured records. In this embodiment, structured data generated by different business systems have differences in field names and definitions. For example, rated_load represents rated load in the equipment and facility ledger, while rated_capacity represents rated capacity in the monitoring system; close_status represents the completion status in the hazard rectification form, while verify_result represents the re-inspection conclusion in the inspection system. The system reads the corresponding field definition rules from the metadata repository, aligns the above fields to a unified field name, and unifies the data type of the fields to a preset type.

[0056] Furthermore, in the unified unit processing, for environmental monitoring data where dust concentration may exist in two calibers (mg / m3 and μg / m3), the system performs unit conversion on the values ​​according to the field caliber rules and records the conversion mark; for equipment parameters where length may exist in two calibers (m and mm), the same conversion is performed according to the rules. In missing value processing, records with missing required fields are marked with a missing type, and are processed according to rules using methods such as source-side completion, backfilling with the most recent valid value of the same object, or setting it as a placeholder for missing values, while retaining the original missing mark for subsequent data quality assessment. In the enumeration mapping processing, status texts from different systems are uniformly mapped to status codes. For example, "Hazard rectified," "Rectification completed," and "Pending re-inspection" are uniformly mapped to preset rectification status codes, and work types "Work at height" and "Work at height" are uniformly mapped to the same work type code, thus obtaining standardized structured records that can be consistently retrieved and statistically analyzed across systems.

[0057] Furthermore, the system extracts the acquisition point identifiers and acquisition time information from the unstructured data, and determines spatial location markers based on the mapping relationship between the acquisition point identifiers and the regional grid. Frame-level timestamp calibration is performed on the on-site video data, sampling time recalibration is performed on the sensor time-series data, and grid coordinate registration is performed on the risk map data to generate multimodal index records aligned with a unified time reference. In this embodiment, the acquisition point identifier can be a camera location number, sensor gateway number, risk map publishing terminal number, or mobile terminal number. The system reads the mapping relationship between acquisition points and the regional grid from the metadata repository. For example, CAM03 covers regional grids G12 and G13, the dust sensor SEN07 is installed in regional grid G12, and the risk map publishing terminal MAP01 uses a grid coding system equivalent to the regional grid. Based on this mapping relationship, the system generates spatial location markers for each video segment, each sensor time-series segment, and each risk map publishing. The spatial location markers at least include the corresponding regional grid object identifier.

[0058] Furthermore, in the frame-level timestamp calibration process, the system obtains the time deviation between the camera's local time and a unified time reference. This deviation can originate from time synchronization records or periodic verification records. When camera time drift is detected, the system corrects the video frame timestamps according to the deviation and records the correction parameters in the frame-level index. In the sampling time recalibration process, to address potential packet loss, retransmission, or sampling interval jitter in sensor time-series data, the system recalibrates the sampling point times using the acquisition time as the anchor point. If necessary, it interpolates missing intervals and marks the interpolation intervals to ensure that the time-series data can be aligned with the video and structured events according to a unified time reference. In the grid coordinate registration process, if the risk map uses planar image coordinates or local grid coordinates, the system maps them to the regional grid object identification system through pre-configured registration points or grid transformation parameters, thereby generating a unified risk map index record. Finally, the multimodal index record includes, but is not limited to, acquisition point identifiers, acquisition times, spatial location markers, and reference information pointing to unstructured content payloads.

[0059] Furthermore, based on the object identifier mapping table, object references in the standardized structured records and multimodal index records are unified into a global object identifier, and a unified time stamp associated with the global object identifier is generated using a unified time base. In this embodiment, the standardized structured records typically include personnel identifiers, equipment identifiers, area identifiers, or task identifiers. For example, a work permit record includes a work permit number and the responsible worker's number, an equipment maintenance record includes an asset number, and a hazard record includes the discoverer's identifier and the hazard location code. The multimodal index records may also include equipment serial numbers, camera coverage area codes, or sensor gateway numbers.

[0060] In some examples, the system uses an object identifier mapping table to replace these source-side identifiers with global object identifiers. For instance, it maps employee number A102, location tag TAG-88, and real-name registration number RID-102 to the same person's global object identifier; it maps asset number TC-001, serial number SN-9A7B, and QR code QR-TC001 to the same device's global object identifier; and it maps area code AREA-12 to the area grid's global object identifier. The generation of unified timestamps is based on a unified time benchmark. The system converts the business time, entry time, or manual reporting time of structured records into unified timestamps and associates and saves them with the corresponding global object identifiers, enabling subsequent aggregation of cross-modal evidence along the timeline based on object identifiers.

[0061] Furthermore, an evidence object is generated according to a preset evidence encapsulation structure. The evidence object includes a content payload, a data source identifier, a unified time stamp, a spatial location marker, an associated global object identifier, and an integrity summary. In this embodiment, the content payload can be a set of fields from a structured record, or a video clip reference, image reference, time-series data clip reference, or risk map snapshot reference. The encapsulation of the evidence object follows the same structure: it at least includes a data source identifier, a unified time stamp, a spatial location marker, and an associated set of global object identifiers, facilitating subsequent retrieval of evidence by object identifier and time range.

[0062] In some examples, integrity summary generation employs a summary strategy tied to the content payload: for structured records, a summary is generated based on key field combinations and includes field caliber version markers; for videos and images, a summary is generated based on content citation identifiers and frame time ranges and is bound to correction parameter markers; for time-series data fragments, a summary is generated based on the sampling point sequence and recalibration parameters; for risk map snapshots, a summary is generated based on grid registration parameters and grid coding sets. The integrity summary, along with the data source identifier, unified timestamp, and global object identifier, is written into the evidence object header information, enabling subsequent verification during consistency check training or online verification to confirm whether the evidence has been tampered with and whether it matches the citation relationships.

[0063] The following is a specific example: Within area grid G12 of project P2026S1, a lifting operation occurred between 10:00 and 10:10 on a certain day. The work permit system generated a structured record, recording the work task identifier PERMIT-20260128-001, the responsible person A102, the work area G12, the work type lifting operation, and the hazard source type lifting operation; camera CAM03 captured video of the lifting operation between 10:02 and 10:08; tower crane sensor gateway SRC02 captured the load and amplitude timing data between 10:00 and 10:10; and the risk map system released a snapshot of the risk level of area G12 at 10:05. The system reads field definition rules and enumeration mapping rules from the metadata repository, and organizes the operation type, hazard source type and status fields into a unified code to obtain standardized structured records. At the same time, it reads the acquisition point mapping of CAM03 and SRC02, determines the spatial location mark as G12, completes the video frame timestamp calibration and time sequence sampling time recalibration, and performs registration on the risk map coordinate system to generate multimodal index records.

[0064] Subsequently, the system maps A102 to a global personnel object identifier, PERMIT-20260128-001 to a global task object identifier, and related equipment identifiers to global equipment object identifiers based on the object identifier mapping table, generating a unified timestamp. Finally, the system encapsulates standardized structured records, video clip references, time-series clip references, and risk map snapshot references into evidence objects. Each evidence object carries a data source identifier, a unified timestamp, a spatial location identifier, an associated global object identifier, and an integrity summary. This allows for subsequent retrieval of cross-modal evidence sets within the 10:00 to 10:10 range based on the global task object identifier, which can then be used for risk classification entry generation or hazard event verification.

[0065] The processing method in this embodiment enables the unification of field definitions and semantic regularization of cross-source structured data based on the metadata repository, and completes the spatiotemporal alignment of unstructured data such as videos, time series, and risk maps under a unified time base and regional grid mapping. Furthermore, it achieves global object identification unification for multi-source object references through an object identification mapping table, and generates evidence objects carrying integrity summaries with a preset evidence encapsulation structure, thereby improving the degree of correlation, retrieval, and verification of multi-source data, and providing a stable and consistent data and evidence foundation for the subsequent construction of six types of libraries and the training of multimodal large models.

[0066] In some embodiments, six types of libraries are constructed based on evidence objects, including: Establish an in-library index structure with the global object identifier as the primary key, and configure evidence identifiers and evidence version tags for evidence objects so that each library entry in the six types of libraries can be referenced and associated through the global object identifier and the evidence identifier; Write the job qualification information, authorization scope information and training record information associated with the personnel object identifier into the personnel information database, and associate the evidence objects corresponding to the job qualification information or authorization scope information into the evidence reference set of the personnel entry; Write the clause objects into the knowledge base according to the clause identifier, establish the association between the clause objects and the operation type and risk type based on the scope of application field of the clause objects, configure the clause version mark for the clause objects, and associate the evidence objects corresponding to the source of the clause objects as evidence references of the clause objects; Write the dynamic risk entries associated with the risk source object identifier into the risk management database. The dynamic risk entries include the risk level and the basis for classification. Write the evidence objects and clause objects referenced in the basis for classification into the reference set of the risk entries. Write the closed-loop link events associated with the hidden danger object identifier into the hidden danger investigation database according to the event time sequence, configure state transition constraints for the closed-loop link events, and associate the evidence objects corresponding to each event as event evidence references; Write the contingency plan version, resource object, and available status of the resource object associated with the scenario object into the emergency management database, and configure the available status update rules and update evidence references for the resource object; The evidence chain and cause chain associated with the accident object identifier are written into the accident management database. The evidence chain is generated by sorting the associated evidence objects based on a unified time stamp, and the cause chain is determined by the accident type and the evidence references in the evidence chain and written into the database entry.

[0067] Specifically, the database index structure refers to the index organization method used to quickly locate database entries in the six types of databases. In this embodiment, a primary index is established using the global object identifier as the primary key, and a time index can be established in combination with a unified time stamp to support joint retrieval by object and time range.

[0068] A clause object refers to a structured clause entity extracted from normative documents, institutional documents, checklist items, or contingency plan clauses. A clause identifier is used to uniquely identify a clause object. A clause version identifier is used to distinguish different versions of a clause object when the document is revised.

[0069] Dynamic risk entries refer to risk record entries that are associated with the risk source object identifier and are updated as the construction stage, environment, or equipment status changes. Dynamic risk entries include risk level and classification basis references.

[0070] A closed-loop event refers to a set of events organized in chronological order during the handling of potential hazards, including at least two of the following: discovery events, dispatch events, rectification events, and re-inspection events. State transition constraints refer to the conditions and order constraints on the state changes of closed-loop events. For example, the state of a hazard cannot be transitioned to closed-loop status without evidence of a re-inspection event.

[0071] A chain of evidence refers to a sequence of evidence objects ordered around an accident object identifier under a unified time marker, used to depict the factual trajectory of key periods before and after the accident. A causal chain refers to a set of causal related entries determined based on the accident type and the evidence references in the chain of evidence, used to solidify the accident's triggering conditions, abnormal signs, and handling process in a structured form.

[0072] In some examples, an in-database index structure is established with a global object identifier as the primary key, and evidence identifiers and evidence version tags are configured for each evidence object. This allows entries in the six categories of databases to be referenced and associated with each other through the global object identifier and the evidence identifier. In this embodiment, the system generates an evidence identifier for each evidence object and binds and stores the evidence identifier with a unified time stamp, spatial location stamp, data source identifier, and integrity summary. When the same data source supplements or corrects evidence objects within the same time range, the system does not overwrite the original evidence object. Instead, it establishes a new evidence version tag with the same evidence identifier and records the version inheritance relationship, so that any subsequent database entry can clearly identify the version when referencing evidence.

[0073] The index structure within the six types of databases uses the global object identifier as the primary key to establish the main index, enabling entries for the same person, equipment, area, task, risk source, hazard, scenario, or accident in different databases to be associated with the same primary key. Simultaneously, the system establishes a unified time-stamped time index for databases requiring time-based retrieval. For example, the risk management database uses a time index based on the risk level update time, the hazard investigation database uses a time index based on the event occurrence time, the emergency management database uses a time index based on the resource status update time, and the accident management database uses a time index based on the accident occurrence time and the time range of the evidence chain.

[0074] Furthermore, the job qualification information, authorization scope information, and training record information associated with the personnel object identifier are written into the personnel information database, and the evidence objects corresponding to the job qualification information or authorization scope information are associated as the evidence reference set for the personnel entry. In this embodiment, the personnel information database uses the global object identifier of the personnel as the primary key, and the fields written include, but are not limited to: job qualification category, qualification validity period, authorization scope, training course identifier, and training completion time. The scanned copies of qualification certificates, training attendance records, and authorization approval records from the real-name system are used as the evidence reference set for the personnel entry. For example, if personnel A102 has a high-altitude operation qualification certificate in the real-name system and completes crane command training on a certain day, the system writes the relevant qualification and training information into the personnel information database and references the qualification certificate image evidence object and the training record evidence object. When this personnel is subsequently assigned as the person in charge of high-altitude operation inspection, the system can directly read the authorization scope from the personnel information database and verify whether its evidence reference set is complete.

[0075] Furthermore, the clause objects are written into the knowledge base according to the clause identifier, and the association between the clause objects and the operation type and risk type is established based on the scope of application field of the clause objects. A clause version tag is configured for each clause object, and the evidence object corresponding to the source of the clause object is associated as the evidence reference of the clause object. In this embodiment, the system performs clause-level segmentation of enterprise safety management systems, standard clauses, checklist items, and emergency plan clauses, extracts the clause identifier, clause text, scope of application, inspection points, and judgment criteria, and writes them into the knowledge base.

[0076] The scope of application includes operation type coding and risk type coding. For example, if a clause applies to lifting and hoisting operations and the risk type is lifting collision risk, then an association will be established in the knowledge base between this clause and the lifting and hoisting operation type and the lifting collision risk type. Clause version markings are used to distinguish changes in clauses before and after policy revisions. For example, if the same clause adds inspection points after an annual revision, the system will generate a new version for the clause while retaining the old version. Simultaneously, the evidence object from which the policy document originated will be used as evidence citation for the clause object, so that the source and version can be traced when the model outputs referenced clauses.

[0077] Furthermore, dynamic risk entries associated with the risk source object identifier are written into the risk management database. Each dynamic risk entry includes a risk level and a grading basis, and the evidence objects and clause objects referenced in the grading basis are written into the reference set of the risk entry. In this embodiment, the risk management database uses the global risk source object identifier or the risk source object identifier as the primary key to write dynamic risk entries. The risk level in a dynamic risk entry can be determined comprehensively based on risk map snapshots, work task status, equipment status, and environmental indicators, and the grading basis must reference evidence objects and clause objects.

[0078] For example, when lifting and hoisting operations are carried out in area G12, the system obtains video clips, tower crane load time sequence clips, and risk map snapshots from the evidence objects, retrieves clause objects applicable to lifting and hoisting operations from the knowledge base, determines the risk level based on the classification rules, and writes it into the risk management database. At the same time, the above evidence object identifiers and clause identifiers are written into the reference set of risk entries, so that subsequent adjustments to the risk level have a traceable evidence chain and normative basis.

[0079] Furthermore, the closed-loop events associated with the hazard object identifier are written into the hazard investigation database according to the event sequence, and state transition constraints are configured for the closed-loop events. The evidence objects corresponding to each event are also associated as event evidence references. In this embodiment, the hazard investigation database uses the global hazard object identifier as the primary key and establishes an event sequence using a unified time stamp.

[0080] In some examples, a discovery event includes the discovery time, location, discoverer's identifier, and hazard type, and references inspection photos or video screenshots as evidence. A dispatch event includes the responsible person's identifier and rectification deadline, and references the dispatch approval document as evidence. A rectification event includes rectification measures and completion time, and references before-and-after photos as evidence. A re-inspection event includes the re-inspector's identifier and re-inspection conclusion, and references the re-inspection record as evidence. State transition constraints are used to regulate the transition conditions of a hazard's state from "discovered" to "dispatched," "rectified," "re-inspected," and finally "closed-loop." For example, a hazard cannot transition to "closed-loop" if there is no re-inspection event evidence, and it cannot transition to "rectified" if the rectification deadline has expired and no extension approval evidence is provided. Through this mechanism, the hazard closed-loop chain can form an auditable process trajectory.

[0081] Furthermore, the contingency plan version, resource objects, and availability status of the resource objects associated with the scene objects are written into the emergency management database, and availability status update rules and update evidence references are configured for the resource objects. In this embodiment, the emergency management database uses the scene object or contingency plan object as the primary key and writes the contingency plan version and resource object list. Resource objects include, but are not limited to, fire extinguishers, first aid kits, stretchers, walkie-talkies, emergency lighting, rescue vehicles, and emergency personnel teams, and the availability status and availability status update time are recorded for each resource object.

[0082] In some examples, availability status update rules can include periodic inspection triggers, post-use return triggers, or expiration replacement triggers, requiring each status update to be associated with updated evidence references, such as inspection photo evidence objects, requisition and return record evidence objects, or replacement approval evidence objects. This way, when the inference model generates an emergency resource allocation sequence, it can directly reference resource objects in the emergency management database that are available and have complete evidence references, avoiding the allocation of unavailable resources.

[0083] Furthermore, the evidence chain and cause chain associated with the accident object identifier are written into the accident management database. The evidence chain is generated by sorting associated evidence objects based on a unified time stamp, while the cause chain is determined by the accident type and the evidence references within the evidence chain and written into the database entry. In this embodiment, when an accident event occurs, the system uses the accident object identifier as the primary key to retrieve accident-related evidence objects from a preset time window before and after the accident. This includes video clips of the accident area, the location sequence of relevant personnel, the equipment status sequence, evidence of closed-loop events related to potential hazards, and risk entry evidence references in the risk management database. The system then generates an evidence chain based on the unified time stamp. Subsequently, the system reads the corresponding cause analysis template clause object from the knowledge base according to the accident type, and extracts triggering conditions, abnormal symptoms, and clues to the handling process by combining the evidence references in the evidence chain. This forms cause chain entries and is written into the accident management database. Simultaneously, the system retains the set of evidence object identifiers referenced by the cause chain entries, enabling the structured accumulation of accident debriefings for subsequent training sample generation.

[0084] The following is a specific example: During lifting operations in area G12 of project P2026S1, the system generates video evidence objects, tower crane load time sequence evidence objects, and risk map evidence objects related to the operation task. The system writes the qualification certificate evidence object and training record evidence object of the person in charge of the operation, A102, into the personnel information database; it writes the inspection clauses applicable to lifting operations into the knowledge base and establishes a relationship with the operation type and risk type; it writes the dynamic risk items of area G12 into the risk management database and references the video evidence objects, time sequence evidence objects, and clause objects in the classification basis; if the warning line of the lifting area is found to be missing during the inspection, the system creates a hidden danger object and writes it into the hidden danger investigation database according to the event time sequence. The discovery event references the inspection photo evidence object, the dispatch event references the dispatch approval evidence object, the rectification event references the rectification post-rectification photo evidence object, and the re-inspection event references the re-inspection record evidence object, and the status migration is strictly constrained; at the same time, the emergency management database records the corresponding emergency plan version and available emergency walkie-talkies, first aid kits, and other resource objects in the area, and references the inspection evidence object when the resource status is updated. If a minor collision occurs subsequently, the system uses the accident object identifier as the primary key to retrieve relevant evidence objects within the time window before and after the accident, generates an evidence chain according to a unified time marker, and combines it with the accident type template clause objects in the knowledge base to generate a cause chain and write it into the accident management database, realizing the comprehensive accumulation from risk and hidden danger to accident review.

[0085] It should be noted that the above six types of databases (personnel information database, knowledge base, risk management database, hazard identification database, emergency management database, and accident management database) are merely preferred embodiments. In specific implementations, the embodiments of this application are not limited to the above six types of databases. For example, they may also include: laws and regulations database, training management database, equipment and facilities database, site management database, continuous improvement database, environmental management database, occupational health management database, etc.; and these databases can be further divided into various functions. For example, the equipment and facilities database may include the following functions: equipment files, intelligent inspection, equipment repair, equipment maintenance, etc.; the site management database may include the following functions: major hazard sources, contractor management, change management, work permits, personnel location, JSA management, etc.; the risk management database may include... The system includes the following functions: risk identification, risk scoring, risk statistics, risk mapping, and risk warning; the hazard identification database can include the following functions: hazard snapshot, random inspection, regular inspection, hazard rectification, hazard acceptance, and hazard statistics; the emergency management database can include the following functions: emergency organization, emergency resources, emergency plans, emergency drills, and emergency response; the accident management database can include the following functions: accident registration, accident investigation, accident sharing, and accident reporting; the continuous improvement database can include the following functions: safety audit, safety warning, and incentive mechanisms; the environmental management database can include the following functions: solid waste management, wastewater management, and waste gas management; and the occupational health management database can include the following functions: occupational hazard factors, PPE management, and physical examination management.

[0086] It should be understood that the types and functions of the above-mentioned libraries are merely examples of this application and do not constitute a limitation on the technical solutions of this application. Those skilled in the art can set up different types of libraries based on actual needs and implement different functions.

[0087] The processing method in this embodiment enables the establishment of a unified index thread for six types of libraries using global object identifiers, and achieves consistency and traceability of evidence citation across library entries through evidence identifiers and evidence version tags. At the same time, it structures and interreferences the evidence chain of personnel qualifications, standard clauses, dynamic risks, hidden danger closure loops, emergency resource status, and accident review, so that the data of safety management in the construction process can form a searchable, auditable, and reusable set of knowledge and facts. This provides a stable source of training samples and evidence citation foundation for subsequent model training and online inference, and improves the credibility and consistency of risk classification, hidden danger closure loops, and emergency deployment at the data level.

[0088] In some embodiments, using evidence objects and entries associated with object identifiers in six categories of libraries as training samples, multimodal representation alignment training and consistency verification training based on relational constraints are performed to obtain a large multimodal model for safety management of the building construction process, including: Based on the global object identifier, a cross-modal evidence set associated with the same object identifier is extracted from the evidence object, and a set of library entries associated with the object identifier is extracted from six types of libraries. The cross-modal evidence set and the set of library entries are aligned according to a unified time stamp to generate a training sample package. Multimodal representation alignment training is performed on the training sample package, including encoding cross-modal evidence payloads into multimodal representation vectors and performing contrastive learning updates on the multimodal representation vectors based on cross-modal consistency constraints under the same global object identifier, so as to obtain a unified semantic representation for construction safety elements; Based on relational constraints, a consistency check is performed on object relational references and clause object references in the training sample package to identify conflicting references that do not meet the relational constraints and generate conflict type flags; the conflict type flags are associated with the corresponding training sample packages as verification samples. Consistency verification training is performed on the verification samples, including limiting the structured output of the multimodal large model to an output format that includes evidence identifier references and object relation references, and performing constraint loss update on the reference consistency in the structured output based on the conflict type label, so that the multimodal large model satisfies relation constraints when generating structured output; After completing the consistency verification training, a multimodal large model for safety management of the building construction process is obtained, and the model version tag of the multimodal large model is associated and stored with the evidence version tag and library entry version tag used to generate the training sample package.

[0089] Specifically, a cross-modal evidence set refers to a set of multimodal evidence objects corresponding to the same global object identifier within a preset time range. Multimodal evidence can include structured records, video clip references, image references, sensor time-series clip references, and risk map snapshot references. Cross-modal consistency constraints refer to rules used to ensure that different modal representations under the same global object identifier are close to each other in the semantic space, and that representations under different object identifiers are distinct from each other.

[0090] Object relationship references refer to reference information in the training sample package used to represent relationships between objects, such as personnel's job positions, work tasks associated with hazards, and event sequences associated with hazard handling links. Clause object references refer to reference information in the training sample package regarding the identifiers and version tags of clause objects in the knowledge base, used to introduce normative constraints into the training. Conflicting references refer to conflicting items that arise when object relationship references or clause object references do not satisfy the set of relationship constraint rules.

[0091] In some examples, a cross-modal evidence set associated with the same object identifier is extracted from the evidence object based on the global object identifier, and a set of library entries associated with the object identifier is extracted from six categories of libraries. The cross-modal evidence set and the set of library entries are aligned according to a unified time stamp to generate a training sample package. In this embodiment, the system uses the global object identifier as the index key to perform sample construction for each type of target object.

[0092] For a work task object, the system retrieves evidence objects within the planned start and end time range of the work task, obtaining a cross-modal evidence set including work permit structured record evidence objects, area video clip evidence objects, equipment status time sequence evidence objects, and risk map snapshot evidence objects. At the same time, it extracts a set of clause objects corresponding to the work type from the knowledge base, extracts dynamic risk entries corresponding to the hazard source from the risk management library, and extracts hazard event entries that occurred within the time range and are related to the work area from the hazard investigation library. The clause object reference, risk level or hazard status field, and object relationship field are used as components of the library entry set.

[0093] The alignment process uses a unified time stamp as a benchmark, aggregates evidence objects by time window and aligns them with the event time field in the library entries. For example, video clips and sensor clips are mapped to the time range of work permit records by time period. Hazard discovery and re-inspection events are bound to the same training sample package in chronological order, so that the training sample package forms an encapsulated structure of "same object - same time axis - multimodal evidence - library entry reference".

[0094] Furthermore, multimodal representation alignment training is performed on the training sample package, including encoding cross-modal evidence payloads into multimodal representation vectors, and performing contrastive learning updates on the multimodal representation vectors based on cross-modal consistency constraints under the same global object identifier, to obtain a unified semantic representation for construction safety elements. In this embodiment, the multimodal evidence payload includes structured field sequences, video frame sequence references, time-series data sequence references, and risk map grid feature references.

[0095] In some examples, the system employs mutually alignable encoding strategies for different payload types: structured field sequences are serialized and encoded according to field caliber rules; video frame sequences are sampled using frame-level timestamps to form segment representations; sensor time series sequences are recalibrated based on sampling times to form segment representations; and risk map snapshots are registered using grid features to form segment representations. The resulting multimodal representation vectors are projected into the same semantic space.

[0096] Contrastive learning updates use different modal segments of the same global object identifier within the same time window as positive sample pairs. For example, a work permit record for the same task and a video segment of the corresponding time period are used as positive sample pairs, as are video segments of the same task and corresponding tower crane load time-series segments. Conversely, segments with different global object identifiers or from different time windows are used as negative sample pairs, such as video segments from different tasks. Through these updates, the model learns a unified semantic representation that can aggregate across modalities, providing a representational foundation for subsequent structured inference output.

[0097] Furthermore, based on relational constraints, a consistency check is performed on the object relation references and clause object references in the training sample package to identify conflicting references that do not satisfy the relational constraints and generate conflict type markers; the conflict type markers are then associated with the corresponding training sample packages as verification samples. In this embodiment, the system performs verification on the training sample packages according to the aforementioned set of relational constraint rules.

[0098] For example, when a training sample package records that a person belongs to two job objects under the same unified time marker, a cardinality exceedance conflict is triggered, and a corresponding conflict type marker is generated; when an emergency resource object is associated with a scene object that does not match it in the training sample package, a category mismatch conflict is triggered; when a hazard handling link lacks a verification event evidence reference but is marked as closed, a link missing conflict is triggered; when the scope of application of a clause object reference is inconsistent with the job type in the training sample package, a scope of application conflict is triggered. The system marks the training sample package with the conflict as a verification sample and retains the conflict reference location, conflict type marker, and object relationship references and clause object references involved in the conflict as a supervision signal for subsequent consistency verification training.

[0099] Furthermore, consistency verification training is performed on the verification samples. This includes limiting the structured output of the multimodal large model to an output format that includes evidence identifier references and object relationship references, and updating the constraint loss based on the reference consistency in the structured output according to the conflict type label, so that the multimodal large model satisfies the relation constraints when generating structured output. In this embodiment, the structured output format includes two parts: a reasoning conclusion field and a reference field. The reference field includes evidence identifier references and object relationship references, so that when the model outputs risk level, hazard status, or emergency resource allocation results, it simultaneously outputs the evidence objects on which they are based and the object relationships involved.

[0100] The system maps conflict type labels to penalty terms in the constraint loss for each validation sample. For example, the penalty term increases when the object relation references in the model output violate cardinality constraints; it increases when the model output references inapplicable clause object identifiers; and it increases when the model outputs a closed-loop state but lacks a reference to a verification event evidence identifier. By updating the constraint loss, the model is trained to automatically avoid reference combinations that do not satisfy relation constraints when generating structured outputs and tends to select output structures that satisfy constraints and have complete evidence references.

[0101] Furthermore, after completing the consistency verification training, a multimodal large model for construction process safety management is obtained, and the model version tag of the multimodal large model is associated and stored with the evidence version tag and library entry version tag used to generate the training sample package. In this embodiment, before the model is released, the system records the version range of the evidence objects used for training, the version range of the six types of library entries, and the version of the safety element ontology and relation constraint rule set, and binds them with the model version tag so that when anomalies in the model output are subsequently discovered, the training data version can be traced and rolled back or incrementally revised. At the same time, when new versions of evidence objects are generated due to timestamp correction or field definition rule updates, the system can regenerate the training sample package based on the version differences and trigger incremental training, so that the model iterates with the evolution of construction process data.

[0102] The following is a specific example: Within the regional grid G12 of project P2026S1, there exists a work task PERMIT-20260128-001. Within the time window of 10:00 to 10:10, the system aggregates structured evidence objects of the work permit, video evidence objects, tower crane load time-series evidence objects, and risk map evidence objects. It also extracts lifting operation clause objects from the knowledge base, dynamic risk entries for the area from the risk management database, and missing warning line hazard events within the same time window from the hazard investigation database, forming a training sample package. During the alignment training phase, the system uses work permits and video clips, and video clips and time-series clips as positive sample pairs, and clips from work tasks in other areas as negative sample pairs to update the multimodal representation vector.

[0103] The system then discovered that the hazard entries in the sample package were marked as closed-loop but lacked supporting evidence for verification events, triggering a link missing conflict and generating a conflict type flag. This sample package was then used as a verification sample. During the consistency verification training phase, the model was constrained to simultaneously output supporting evidence for verification events when outputting the closed-loop status of the hazards; otherwise, the constraint loss would increase, prompting the model to adjust its output structure to more readily output reference combinations that conformed to the link constraints. The final generated model version was associated with the evidence version and the library entry version for storage, making the training basis of the model traceable.

[0104] The processing method in this embodiment can encapsulate cross-modal evidence and six types of library entries into an alignable training sample package using global object identifiers as the main thread, and obtain a unified semantic representation for construction safety elements through comparative learning. At the same time, relational constraints are introduced to verify the consistency of object relation references and clause object references, and the consistency verification training is driven by conflict type tags. This ensures that the model output has evidence identifier references and object relation references and satisfies relational constraints, thereby improving the auditability of the model training samples and the consistency of the output structure. This provides a stable and reliable model foundation for risk classification, hidden danger closed-loop judgment, and emergency resource allocation in the subsequent online inference stage.

[0105] In some embodiments, the multimodal large model is deployed as a reasoning model for risk classification, hazard closure determination, and emergency resource allocation based on evidence object references, including: Configure the inference input structure for the multimodal large model. The inference input structure includes the target global object identifier, the query time range and the evidence retrieval conditions. Based on the target global object identifier, read the object association entries, clause object references and relation constraints from six types of libraries, and read the cross-modal evidence payloads that match the query time range from the evidence objects to generate the inference context. Input the reasoning context into the multimodal large model and output the structured reasoning results, which carry evidence identification references and object relationship references. Based on relational constraints, online consistency checks are performed on evidence identifier references and object relation references in the structured reasoning results. When the checks pass, the structured reasoning results are written into the risk management database, the hidden danger investigation database, and / or the emergency management database. When the online consistency check fails, supplementary evidence retrieval or clause object retrieval is triggered on the reasoning context based on the conflict type, and the supplemented reasoning context is re-input into the multimodal large model to generate updated structured reasoning results; The structured reasoning results that have undergone online consistency verification are encapsulated into result messages and output to the construction process safety management business system for risk classification and handling, hazard dispatch and rectification, or emergency resource allocation.

[0106] Specifically, the inference input structure refers to the standardized input data structure received by the inference model, including the target global object identifier, the query time range, and the evidence retrieval conditions, which are used to determine the target object and the scope of evidence around which this inference revolves.

[0107] The reasoning context refers to the set of contextual data extracted and aggregated from the six types of libraries and the set of evidence objects based on the reasoning input structure. The reasoning context includes, but is not limited to: object association entries, clause object references, relationship constraints, and cross-modal evidence payloads.

[0108] Structured reasoning results refer to the structured results output by the multimodal large model that can be parsed by the business system. They include reasoning conclusion fields and reference fields, with reference fields including evidence identification references and object relationship references.

[0109] Online consistency verification refers to the process of performing real-time verification of evidence identifier references and object relationship references based on relational constraints before the inference results are written into the database. This is used to prevent results that do not meet the constraints from being written into the six types of databases. Conflict type refers to the classification label for the reasons for conflict when online consistency verification fails, which is used to drive supplementary retrieval strategies.

[0110] Supplementary evidence retrieval refers to the process of expanding the evidence retrieval criteria to retrieve more evidence objects when insufficient evidence citations or mismatched evidence versions are found. Supplementary clause object retrieval refers to the process of expanding the clause retrieval criteria to reselect clause objects when inapplicable or missing clause object citations are found.

[0111] The result message refers to the message carrier that encapsulates the reasoning results and is used for inter-system interaction. It includes result identifier, target global object identifier, model version marker, evidence version marker, structured reasoning results, and verification status information.

[0112] In some examples, an inference input structure is configured for the multimodal large model. This inference input structure includes a target global object identifier, a query time range, and evidence retrieval conditions. Based on the target global object identifier, object association entries, clause object references, and relational constraints are read from six categories of libraries. Cross-modal evidence payloads matching the query time range are read from the evidence objects to generate the inference context. In this embodiment, the target global object identifier can be a task global object identifier, a risk source global object identifier, a hazard global object identifier, a regional grid global object identifier, or a scenario global object identifier. The query time range is used to limit the evidence window used for inference, such as the planned start and end time range around the current control cycle, the current shift, or a specific work permit. The evidence retrieval conditions are used to limit the filtering conditions for evidence objects, including a set of data source identifiers, a spatial location marker range, evidence version preferences, and evidence type preferences, such as prioritizing video evidence objects with calibrated timestamps and recalibrated sensor evidence objects.

[0113] In some implementations, when generating the reasoning context, the system first reads object-related entries from six categories of libraries based on the target global object identifier. For example, it reads personnel qualifications and authorization scope from the personnel information library, clause object references associated with operation type and risk type from the knowledge library, historical risk entries from the risk management library, unclosed hazard links from the hazard investigation library, scenario plan versions and resource availability status from the emergency management library, and evidence chain indexes of similar accidents from the accident management library. At the same time, it reads the set of relational constraint rules associated with the target object to limit the relational structure of the reasoning output.

[0114] Subsequently, based on the query time range and evidence retrieval conditions, the system retrieves cross-modal evidence payloads from the evidence object set, such as structured inspection record evidence objects, video clip evidence objects, dust and noise time-series evidence objects, equipment load time-series evidence objects, and risk map snapshot evidence objects. The system also retains the evidence identifier, evidence version mark, and integrity summary of each evidence object in the reasoning context for subsequent citation and verification.

[0115] Furthermore, the reasoning context is input into the multimodal large model, and a structured reasoning result is output, carrying evidence identifier references and object relationship references. In this embodiment, the structured reasoning result outputs different conclusion fields according to the business type, and uniformly carries reference fields. For risk classification business, the conclusion fields include the risk level and key points of classification basis, and the reference fields include the evidence identifier reference set and clause object reference set used to support the risk level, as well as object relationship references between risk sources and work tasks, regional grids, etc.; for hidden danger closed-loop judgment business, the conclusion fields include the hidden danger status and link missing prompts, and the reference fields include, but are not limited to: evidence identifier references corresponding to events such as discovery, rectification, and re-inspection, as well as object relationship references between hidden dangers and event sequences; for emergency resource allocation business, the conclusion fields include the resource allocation sequence and resource object selection results, and the reference fields include evidence identifier references corresponding to the resource availability status and scenario plan clause object references, as well as object relationship references between scenario objects and resource objects. Through the above methods, the reasoning result itself carries traceable evidence references and relationship references, enabling the business system to verify and audit the reasoning conclusions.

[0116] Furthermore, based on relational constraints, online consistency checks are performed on the evidence identifier references and object relation references in the structured reasoning results. Upon successful verification, the structured reasoning results are written into at least one of the risk management database, the hidden danger investigation database, and the emergency management database. In this embodiment, online consistency checks include two types: evidence reference verification and relational structure verification. Evidence reference verification checks whether the evidence identifier reference exists, whether the evidence version marker is within the allowed range, and whether the integrity summary of the evidence object is consistent with the reference binding information, thus avoiding evidence that references non-existent or expired evidence.

[0117] In some implementations, relational structure verification is used to check whether object relation references meet the restrictions on the object categories that the relation can connect to, the cardinality of the relation, and the effective time range of the relation. For example, it can check whether the work task referenced by the risk item has an association with the risk source and whether the time range covers the query time range; it can check whether there is a re-verification event evidence identifier reference when the hazard closure status is closed; and it can check whether the selected resource object in the emergency resource allocation sequence is available in the emergency management database and whether there is a corresponding update evidence reference.

[0118] Furthermore, after the verification is passed, the system writes the structured reasoning results into the corresponding database entries according to the business type. For example, the risk level and classification basis are written into the dynamic risk entry in the risk management database, the hidden danger status update is written into the link event entry in the hidden danger investigation database, and the resource allocation record is written into the allocation record entry in the emergency management database. At the same time, the model version mark and evidence version mark are recorded for subsequent backtracking.

[0119] In some examples, when online consistency verification fails, supplementary evidence retrieval or clause object retrieval is triggered on the inference context based on the conflict type. The supplemented inference context is then re-input into the multimodal large model to generate updated structured inference results. In this embodiment, the system selects different supplementation strategies based on the conflict type. If the conflict type is missing evidence or mismatched evidence versions, supplementary evidence retrieval is triggered, such as expanding the boundaries of the query time range, relaxing evidence type preferences, increasing the data source identifier set, or expanding the spatial location marker range to retrieve more evidence objects and add them to the inference context. If the conflict type is a conflict in the scope of application of clauses, supplementary clause object retrieval is triggered, such as re-filtering the clause object versions in the knowledge base based on the operation type, risk type, and on-site management strategy, and adding the updated clause object references to the inference context. If the conflict type is a conflict in relation cardinality or relation category, object relation completion and reconstruction are triggered, such as supplementing missing object association entries from the six-category library or adjusting the candidate set of object relation references, and adding the adjustment results to the inference context. After the supplement is completed, the system will input the updated inference context back into the multimodal large model to generate new structured inference results, and perform online consistency verification again until the verification passes or the preset retry limit is reached.

[0120] Furthermore, the structured reasoning results verified online are encapsulated into a result message and output to the construction process safety management system for risk classification and handling, hazard dispatch and rectification, or emergency resource allocation. In this embodiment, the result message includes, but is not limited to: result identifier, target global object identifier, query time range, model version marker, evidence version marker, structured reasoning result, evidence identifier reference list, object relationship reference list, and online consistency verification status.

[0121] Furthermore, after receiving the result message, the business system triggers corresponding business actions based on the structured reasoning results: in the risk classification and handling scenario, a risk handling task is generated and associated with the evidence reference of the classification basis; in the hidden danger dispatch and rectification scenario, a dispatch event is generated and associated with the dispatch event with the hidden danger link and event evidence reference; in the emergency resource allocation scenario, a resource allocation instruction is generated and associated with the allocation result with the evidence reference of the resource availability status, thereby realizing the closed-loop linkage from reasoning results to business execution.

[0122] The following is a specific example: Within the regional grid G12 of project P2026S1, there is currently a lifting and hoisting operation corresponding to the global object identifier of the task. The system uses this task global object identifier as the target global object identifier, takes 10:00 to 10:10 as the query time range, and prioritizes the retrieval of CAM03 video evidence object, SRC02 load time sequence evidence object, and risk map evidence object as evidence retrieval conditions. It generates a reasoning context, reads the clause object reference of the lifting and hoisting operation from the knowledge base, reads the qualification of the responsible person from the personnel information database, and reads the version of the contingency plan for the hoisting collision scenario and the availability status of available emergency walkie-talkies and first aid kits from the emergency management database.

[0123] Furthermore, the multimodal large model outputs structured inference results, providing risk levels and referencing corresponding video evidence identifiers and load timing evidence identifiers. It also outputs object relationship references between the task, hazard source, and regional grid. Online consistency verification detects that the clause object version referenced by the risk level is not within the current effective scope, generating a clause scope conflict type. The system triggers a supplementary clause object retrieval, reselects the effective version of the clause object reference, updates the inference context, and after another inference, the verification passes. The risk entry is written to the risk management database, and the result message is encapsulated and output. The business system generates risk disposal tasks based on this, attaching evidence references to the tasks. Subsequently, users can directly click on the evidence references to view the corresponding video clips and timing segments.

[0124] The processing method in this embodiment can standardize the inference input of a multimodal large model into an inference context that includes the target global object identifier and time range, and carry evidence identifier references and object relationship references in the inference output; at the same time, it uses relation constraints to perform online consistency verification and triggers supplementary retrieval to form a closed loop correction when there is a conflict, so that the inference results can be stably written into the risk management database, hidden danger investigation database and emergency management database and directly executed by the business system, thereby improving the traceability, auditability and reliability of online handling of risk classification, hidden danger closed loop and emergency resource allocation.

[0125] The following are system embodiments of this application, which can be used to execute the method embodiments of this application. For details not disclosed in the system embodiments of this application, please refer to the method embodiments of this application.

[0126] Figure 2 This is a structural schematic diagram of the multimodal large-scale model construction system for building construction process safety management provided in this application embodiment. For example... Figure 2 As shown, the system includes: The acquisition module 201 is used to acquire structured and unstructured data of safety management in the construction process, and construct a safety element ontology. Based on the safety element ontology, global object identifiers and relational constraints are assigned to objects in the structured and unstructured data. The generation module 202 is used to perform field normalization, semantic mapping and spatiotemporal alignment processing on structured and unstructured data, and generate evidence objects based on global object identifiers and unified time stamps; Module 203 is used to construct six types of libraries based on evidence objects, including a personnel information library, a knowledge library, a risk management library, a hazard investigation library, an emergency management library, and an accident management library. Among them, the personnel information library stores the job qualifications and authorization scope associated with personnel object identifiers; the knowledge library stores the association between clause objects and job types and risk types; the risk management library stores dynamic risk entries and classification criteria associated with risk source object identifiers; the hazard investigation library stores closed-loop chain events associated with hazard object identifiers; the emergency management library stores the plan version and resource object availability status associated with scenario objects; and the accident management library stores the evidence chain and cause chain associated with accident object identifiers. Training module 204 is used to perform multimodal representation alignment training and consistency verification training based on relational constraints using evidence objects and entries associated with object identifiers in six categories of libraries as training samples, to obtain a multimodal large model for safety management of building construction process, and to deploy the multimodal large model as a reasoning model for risk classification, hidden danger closed-loop judgment and emergency resource allocation based on evidence object references.

[0127] Figure 3 This is a schematic diagram of the electronic device 3 provided in an embodiment of this application. Figure 3 As shown, the electronic device 3 of this embodiment includes a processor 301, a memory 302, and a computer program 303 stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program 303, it implements the steps in the various method embodiments described above. Alternatively, when the processor 301 executes the computer program 303, it implements the functions of each module / unit in the system embodiments described above.

[0128] Electronic device 3 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 3 may include, but is not limited to, processor 301 and memory 302. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or different components.

[0129] The processor 301 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0130] The memory 302 can be an internal storage unit of the electronic device 3, such as a hard disk or memory of the electronic device 3. The memory 302 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 3. The memory 302 can also include both internal and external storage units of the electronic device 3. The memory 302 is used to store computer programs and other programs and data required by the electronic device.

[0131] The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium (e.g., a computer-readable storage medium). Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, it can implement the steps of the various method embodiments described above. The computer program can include computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable storage medium can include: any entity or device capable of carrying computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc.

[0132] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for constructing a multimodal large-scale model for safety management during building construction, characterized in that, include: Obtain structured and unstructured data for safety management during building construction, construct a safety element ontology, and assign global object identifiers and relational constraints to objects in the structured and unstructured data based on the safety element ontology; The structured and unstructured data are processed by field normalization, semantic mapping and spatiotemporal alignment, and evidence objects are generated based on the global object identifier and unified time stamp. Based on the aforementioned evidence objects, six types of databases are constructed, including a personnel information database, a knowledge base, a risk management database, a hazard investigation database, an emergency management database, and an accident management database. Among them, the personnel information database stores the job qualifications and authorization scope associated with personnel object identifiers; the knowledge base stores the association between clause objects and job types and risk types; the risk management database stores dynamic risk entries and classification criteria associated with risk source object identifiers; the hazard investigation database stores closed-loop events associated with hazard object identifiers; the emergency management database stores the plan versions and resource object availability status associated with scenario objects; and the accident management database stores the evidence chain and cause chain associated with accident object identifiers. Using the evidence objects and the entries associated with the object identifiers in the six categories of libraries as training samples, multimodal representation alignment training and consistency verification training based on the relational constraints are performed to obtain a multimodal large model for safety management of the construction process. The multimodal large model is then deployed as a reasoning model based on risk classification, hazard closed-loop judgment, and emergency resource allocation based on evidence object references.

2. The method according to claim 1, characterized in that, The acquisition of structured and unstructured data for safety management during the construction process, and the construction of a safety element ontology, includes: The system acquires structured and unstructured data for safety management during the construction process, and records data source identifiers and collection time markers for the structured and unstructured data respectively. The structured data includes target and responsibility information, on-site management information, equipment and facility information, environmental information, occupational health information, risk classification information, hidden danger investigation and management information, emergency management information, accident management information, and inspection records. The unstructured data includes on-site video data, sensor time-series data, and risk map data. The safety element ontology is constructed based on a pre-defined set of safety objects and a set of relationship types. Safety objects include personnel, positions, work tasks, regional grids, equipment and facilities, hazard sources, risk levels, control measures, hidden dangers, emergency resources and accident events. Relationship types include subordinate relationships, association relationships, spatiotemporal inclusion relationships and handling link relationships. Relationship constraints are configured for the safety element ontology. For data items in the structured and unstructured data, semantic mapping and object normalization based on the security element ontology are performed to map personnel identifiers, equipment identifiers, area identifiers, or task identifiers in the data items to object identifiers corresponding to security objects.

3. The method according to claim 2, characterized in that, The process of assigning global object identifiers and relational constraints to objects in the structured and unstructured data based on the security element ontology includes: Configure object identification rules for various security objects in the security element ontology. The object identification rules include object category code and field scope code, and generate object fingerprints based on the key attributes of the object to obtain the global object identifier corresponding to the object. For the same object with multiple identification information from different data sources, cross-source deduplication and merging are performed based on the object fingerprint to determine the master object record, and the source identifiers associated with the master object record are written into the object identifier mapping table; A set of relation constraint rules is generated based on the set of relation types of the security element ontology. The set of relation constraint rules includes the object category pairs that the relation can connect to, the cardinality constraint of the relation, and the effective time range constraint of the relation. The global object identifier, the object identifier mapping table, and the set of relationship constraint rules are associated and stored. When writing index records of structured or unstructured data, the objects are uniformly referenced as the global object identifier based on the object identifier mapping table, and constraint verification is performed on the relationships between objects based on the set of relationship constraint rules.

4. The method according to claim 1, characterized in that, The process of performing field normalization, semantic mapping, and spatiotemporal alignment on the structured and unstructured data, and generating evidence objects based on the global object identifier and unified time stamp, includes: The metadata repository reads the field definition rules and enumeration mapping rules associated with each data source identifier. Based on the field definition rules, the structured data is aligned, units are standardized, and missing values ​​are handled. Based on the enumeration mapping rules, the field values ​​are semantically normalized to generate standardized structured records. Extract the collection point identifier and collection time information from unstructured data, and determine the spatial location mark based on the mapping relationship between the collection point identifier and the regional grid; perform frame-level timestamp calibration on the on-site video data, perform sampling time recalibration on the sensor time series data, and perform grid coordinate registration on the risk map data to generate multimodal index records aligned with a unified time reference; Based on the object identifier mapping table, the object references in the standardized structured records and the multimodal index records are unified into the global object identifier, and a unified time stamp associated with the global object identifier is generated using the unified time base. Evidence objects are generated according to a preset evidence encapsulation structure. The evidence objects include content payload, data source identifier, unified time stamp, spatial location stamp, associated global object identifier, and integrity summary.

5. The method according to claim 1, characterized in that, The six types of libraries constructed based on the evidence objects include: Establish an in-library index structure with the global object identifier as the primary key, and configure evidence identifiers and evidence version tags for evidence objects so that each library entry in the six types of libraries can be referenced and associated through the global object identifier and the evidence identifier; Write the job qualification information, authorization scope information and training record information associated with the personnel object identifier into the personnel information database, and associate the evidence objects corresponding to the job qualification information or authorization scope information as the evidence reference set of the personnel entry; Write the clause objects into the knowledge base according to the clause identifier, establish the association between the clause objects and the operation type and risk type based on the scope of application field of the clause objects, configure the clause version mark for the clause objects, and associate the evidence objects corresponding to the source of the clause objects as evidence references of the clause objects; Write the dynamic risk entries associated with the risk source object identifier into the risk management database. The dynamic risk entries include the risk level and the basis for classification. Write the evidence objects and clause objects referenced in the basis for classification into the reference set of the risk entries. Write the closed-loop link events associated with the hidden danger object identifier into the hidden danger investigation database according to the event time sequence, configure state transition constraints for the closed-loop link events, and associate the evidence objects corresponding to each event as event evidence references; Write the contingency plan version, resource object, and available status of the resource object associated with the scenario object into the emergency management database, and configure the available status update rules and update evidence references for the resource object; The evidence chain and cause chain associated with the accident object identifier are written into the accident management database. The evidence chain is generated by sorting the associated evidence objects based on a unified time stamp, and the cause chain is determined by the accident type and the evidence references in the evidence chain and written into the database entry.

6. The method according to claim 1, characterized in that, The process involves using the evidence object and entries associated with the object identifier in the six categories of libraries as training samples to perform multimodal representation alignment training and consistency verification training based on the relational constraints, resulting in a large multimodal model for safety management of the construction process, including: Based on the global object identifier, a cross-modal evidence set associated with the same object identifier is extracted from the evidence object, and a set of library entries associated with the object identifier is extracted from six types of libraries. The cross-modal evidence set and the set of library entries are aligned according to a unified time stamp to generate a training sample package. Multimodal representation alignment training is performed on the training sample package, including encoding cross-modal evidence payloads into multimodal representation vectors and performing contrastive learning updates on the multimodal representation vectors based on cross-modal consistency constraints under the same global object identifier, so as to obtain a unified semantic representation for construction safety elements; Based on relational constraints, a consistency check is performed on object relational references and clause object references in the training sample package to identify conflicting references that do not meet the relational constraints and generate conflict type markers; the conflict type markers are associated with the corresponding training sample packages as verification samples. Consistency verification training is performed on the verification samples, including limiting the structured output of the multimodal large model to an output format that includes evidence identifier references and object relationship references, and performing constraint loss update on the reference consistency in the structured output based on the conflict type label, so that the multimodal large model satisfies the relationship constraints when generating structured output; After completing the consistency verification training, a multimodal large model for safety management of the building construction process is obtained, and the model version tag of the multimodal large model is associated and stored with the evidence version tag and library entry version tag used to generate the training sample package.

7. The method according to claim 1, characterized in that, The deployment of the multimodal large model as a reasoning model based on evidence object references for risk classification, hazard closure judgment, and emergency resource allocation includes: Configure the inference input structure for the multimodal large model. The inference input structure includes the target global object identifier, the query time range and the evidence retrieval conditions. Based on the target global object identifier, read the object association entries, clause object references and relation constraints from six types of libraries, and read the cross-modal evidence payloads that match the query time range from the evidence objects to generate the inference context. The reasoning context is input into the multimodal large model, and the structured reasoning result is output. The structured reasoning result carries evidence identification references and object relationship references. Based on relational constraints, online consistency checks are performed on evidence identifier references and object relation references in the structured reasoning results. When the checks pass, the structured reasoning results are written into the risk management database, the hidden danger investigation database, and / or the emergency management database. When the online consistency check fails, supplementary evidence retrieval or clause object retrieval is triggered on the reasoning context based on the conflict type, and the supplemented reasoning context is re-input into the multimodal large model to generate updated structured reasoning results; The structured reasoning results that have undergone online consistency verification are encapsulated into result messages and output to the construction process safety management business system for risk classification and handling, hazard dispatch and rectification, or emergency resource allocation.

8. A multimodal large-scale model construction system for safety management in building construction processes, characterized in that, include: The acquisition module is used to acquire structured and unstructured data of safety management in the construction process, and construct a safety element ontology. Based on the safety element ontology, global object identifiers and relational constraints are assigned to objects in the structured and unstructured data. The generation module is used to perform field normalization, semantic mapping and spatiotemporal alignment processing on the structured data and unstructured data, and generate evidence objects based on the global object identifier and unified time stamp; The construction module is used to build six types of libraries based on the evidence objects, including a personnel information library, a knowledge library, a risk management library, a hidden danger investigation library, an emergency management library, and an accident management library. Among them, the personnel information library stores the job qualifications and authorization scope associated with the personnel object identifier; the knowledge library stores the association between the clause object and the operation type and risk type; the risk management library stores the dynamic risk entries and classification basis associated with the risk source object identifier; the hidden danger investigation library stores the closed-loop chain events associated with the hidden danger object identifier; the emergency management library stores the plan version and resource object availability status associated with the scenario object; and the accident management library stores the evidence chain and cause chain associated with the accident object identifier. The training module is used to perform multimodal representation alignment training and consistency verification training based on the relational constraints using the evidence object and the entries associated with the object identifier in the six categories of libraries as training samples, to obtain a multimodal large model for safety management of the construction process, and to deploy the multimodal large model as a reasoning model for risk classification, hidden danger closed-loop judgment and emergency resource allocation based on evidence object references.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images