Scene monitoring work rule automatic generation method and system based on big data processing

Abstract

This invention relates to the field of construction management technology, specifically to a method and system for automatically generating scenario-based supervision details based on big data processing. The method includes: processing historical project archives and on-site multimodal perception data; extracting typical construction activities and matching them with current action segments; constructing a construction procedure knowledge graph containing spatiotemporal constraints based on project specifications and design drawings; using this graph to perform compliance analysis on current construction actions, identifying action deviation events and their spatiotemporal coordinates, and tracing back to specific supervision items and safety clauses; integrating the above information to dynamically assemble targeted supervision detail segments, and sorting and connecting segments within consecutive time periods according to time constraints to ultimately form a complete supervision detail document. This method achieves automatic generation from on-site perception data to structured, executable supervision documents, improving the accuracy and systematic nature of construction supervision.

Description

Technical Field

[0001] This invention relates to the field of construction management technology, and in particular to a method and system for automatically generating scenario supervision details based on big data processing. Background Technology

[0002] Currently, construction site supervision mainly relies on manual inspections or video surveillance alarms based on fixed rules. Existing technical solutions typically analyze monitoring data in isolation, lacking an understanding of the inherent procedures and spatiotemporal constraints of construction activities. Its core flaw lies in its inability to place perceived instantaneous events within specific construction procedures, time nodes, and spatial locations for compliance judgment, nor can it automatically associate the event with the detailed regulations and clauses violated, resulting in insufficient operability of alarm information.

[0003] Existing methods struggle to transform discrete anomalies into systematic, executable, and structured supervisory instructions. On-site supervision requires action guidelines based on standards, with clear spatiotemporal orientation and logical sequence. Alarm lists or event logs generated by conventional technologies are fragmented and lack internal coherence; events at different points in time lack temporal and logical connections based on the process flow. This results in a significant gap between massive amounts of sensory data and actionable supervisory documents. Supervisory decisions still heavily rely on manual information integration and text compilation, a cumbersome process that is difficult to standardize.

[0004] A technological approach is needed that can automatically understand the spatiotemporal constraints in construction procedures, accurately correlate on-site action deviations with specific regulations and clauses, and dynamically synthesize fragmented deviation events into a logically coherent and complete supervision document that can directly guide on-site actions. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing an automatic generation method and system for scenario supervision rules based on big data processing.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an automatic generation method for scenario supervision details based on big data processing, comprising: Acquire historical engineering archive data and real-time collected on-site multimodal perception data of the target construction area; The historical engineering archive data is destructured and extracted to identify typical construction activity sequences and corresponding environmental status labels. The real-time collected on-site multimodal sensing data is spatiotemporally aligned and fused to generate a fused sensing data stream, and the current construction action segment that matches the typical construction activity sequence is extracted from the fused sensing data stream. Based on the project specification text and the design drawing data, a construction procedure knowledge graph containing time constraints and space constraints is constructed. The construction procedure knowledge graph is used to perform compliance matching analysis on the current construction action segment to identify action deviation events that deviate from the standard procedure and their spatiotemporal coordinates. Based on the type and spatiotemporal coordinates of the aforementioned action deviation events, the relevant supervision items and safety clauses are traced back and associated in the construction procedure knowledge graph. By integrating the specific parameters of the supervision items, safety clauses, and action deviation events, a supervision detail fragment for the current construction action segment is dynamically assembled and generated; Multiple supervision detail fragments generated within a continuous time window are sorted and logically connected according to the time constraints to form a complete supervision detail document for the target construction area.

[0007] As a further aspect of the present invention, the step of structurally deconstructing the historical engineering archive data to extract typical construction activity sequences and corresponding environmental status labels specifically includes: The project specification text, design drawing data and past construction logs are separated, and typical construction activity sequences and corresponding environmental status labels are extracted based on the past construction logs. Natural language processing technology is used to parse the historical project archive data, identify and separate the project specification text in text format, the design drawing data in image format, and the structured past construction logs; Event extraction is performed on the unstructured text descriptions in the past construction logs to identify each specific construction operation, operation execution time, operation personnel, and associated machinery and materials recorded in the logs; The identified specific construction operations are arranged in chronological order of their execution time to form an initial construction activity sequence; Pattern mining is performed on the initial construction activity sequence to identify construction activity sub-sequences that repeat in multiple historical projects and have fixed operation steps, and these construction activity sub-sequences are marked as typical construction activity sequences. Extract temperature, humidity, light, and wind speed data recorded simultaneously with each operation step in the typical construction activity sequence from the past construction logs, and bind the temperature, humidity, light, and wind speed data as environmental status tags to the corresponding operation steps; A data template is established for each of the typical construction activity sequences. The data template includes a list of operation steps for the typical construction activity sequence and a range of values ​​for the environmental status label bound to each operation step.

[0008] As a further aspect of the present invention, the step of spatiotemporally aligning and fusing the real-time acquired on-site multimodal sensing data to generate a fused sensing data stream, and extracting the current construction action segment matching the typical construction activity sequence from the fused sensing data stream, specifically includes: The system receives real-time sensing data from various types of sensors deployed in the target construction area. The sensing data includes video streams, audio streams, depth images, and IoT sensor readings. Each type of sensing data is accompanied by a timestamp and the location code of the acquisition device. A unified spatiotemporal coordinate system is established, and all categories of the perceived data are mapped to the unified spatiotemporal coordinate system based on the timestamp and the location code to achieve spatiotemporal alignment of multi-source data. In the unified spatiotemporal coordinate system, feature-level fusion is performed on data from different sensors at the same spatiotemporal point. The visual features of the video stream, the acoustic features of the audio stream, the spatial features of the depth image, and the numerical features of the IoT sensor readings are combined into a multi-dimensional feature vector to form the fused sensing data stream. The fused sensing data stream is input into a pre-trained action recognition model. The action recognition model uses the data template of the typical construction activity sequence as a reference to identify and cut out data segments in the fused sensing data stream that match the action patterns of the typical construction activity sequence in real time. Each identified data segment is labeled with the identifier of its corresponding typical construction activity sequence, and the start time, end time, and spatial range covered by the data segment are recorded, thereby generating the current construction action segment.

[0009] As a further aspect of the present invention, the construction of a construction procedure knowledge graph containing time and space constraints based on the project specification text and the design drawing data specifically includes: Deep semantic analysis was performed on the project specification text to identify all key entities and their attributes related to construction procedures, quality requirements, safety standards, and acceptance conditions, as well as the hierarchical, dependency, and causal relationships among these key entities. The design drawing data is parsed to extract the two-dimensional plan layout, three-dimensional structural model, component size information, and key coordinate points of the construction area; The key entities and their relationships obtained from semantic analysis are associated and mapped with the spatial layout and structural information extracted from the design drawings, and each key entity is assigned its corresponding location range and geometric constraints in the actual construction space. Based on the aforementioned association mapping, a time dimension is introduced to parse the preset start time, end time, duration, and temporal logical relationship between each construction procedure from the project specification text, forming a time constraint network. The key entities, entity attributes, semantic relationships between entities, spatial constraint information, and temporal constraint network are integrated and stored and represented using a graph structure. Nodes represent entities or attributes, and edges represent relationships, spatial constraints, or temporal constraints, thereby constructing the construction procedure knowledge graph. In the construction procedure knowledge graph, a status attribute is set for each edge representing a process, which is used to identify the compliance status, progress status, or alarm status of the representative process during real-time monitoring.

[0010] As a further aspect of the present invention, the step of using the construction procedure knowledge graph to perform compliance matching analysis on the current construction action segment, and identifying action deviation events that deviate from the standard procedure and their spatiotemporal coordinates, specifically includes: The specific action type, the subject of the action, the object of the action, and the actual time and spatial coordinates of the action are extracted from the current construction action segment. Using the subject performing the action and the object acting on the action as query conditions, a search is performed in the construction procedure knowledge graph to locate the standard process node that the subject performing the action should perform on the object acting on the action as specified in the graph. Read the attribute information associated with the standard process node, including the description of the standard process operation steps, the standard time window allowed to be executed, the standard spatial area allowed to be executed, the required environmental conditions, and the required tools or materials; The actual time, actual spatial coordinates, and real-time environmental status data extracted from the fused perception data stream are parsed from the current construction action segment and compared one by one with the standard time window, standard spatial region, and required environmental status conditions associated with the standard process node. When the actual time exceeds the standard time window, or the actual spatial coordinates exceed the standard spatial area, or the real-time environmental status data does not meet the required environmental status conditions, an action deviation event is determined to have occurred. The type of the action deviation event is recorded as time deviation, spatial deviation, or environmental condition deviation, and the actual time and actual spatial coordinates corresponding to the action deviation event are recorded as the spatiotemporal coordinates.

[0011] As a further aspect of the present invention, the step of backtracking and associating the relevant supervision items and safety clauses in the construction procedure knowledge graph based on the type and spatiotemporal coordinates of the action deviation event specifically includes: Based on the type of the action deviation event, determine the adjacent process nodes or related entities affected by the deviation type in the construction procedure knowledge graph; Starting from the process node where the action deviation event occurs, trace back along the causal or dependency relationship edges in the construction procedure knowledge graph to find the superior process node or prerequisite node that caused the action deviation. Simultaneously, by searching along the hierarchical relationship edge in the construction procedure knowledge graph, the superior supervision classification node covering the process node is located, and the superior supervision classification node has specific supervision items attached to it. Based on the specific parameters of the action deviation event, retrieve the safety standard entity related to the specific parameters in the construction procedure knowledge graph, and extract the detailed safety clause text linked to the safety standard entity. The traced superior process node or prerequisite node, the retrieved supervision item, and the extracted safety clause text are collectively output as a set of rules related to the action deviation event.

[0012] As a further aspect of the present invention, the integration of the specific parameters of the supervision items, safety clauses, and action deviation events, and the dynamic assembly of the generated supervision detail fragments for the current construction action segment, specifically involves: Establish a supervision rule fragment template library, which contains text generation templates for different types of action deviation events. Each template contains a fixed statement section and variable parameter placeholders. Based on the type of the action deviation event, a corresponding text generation template is selected from the template library; The specific parameters of the action deviation event, including deviation type, occurrence time, occurrence location, and the subject and object involved, are filled into the corresponding variable parameter placeholders of the selected text generation template. Insert the titles and key requirements of the relevant supervision items, as well as the core provisions of the security clauses, into the designated positions in the text generation template in logical order; Using natural language generation technology, the text template after filling and inserting content is processed to polish sentences and ensure logical coherence, generating grammatically correct and clearly expressed supervisory detail text fragments. Add metadata to the generated supervision detail text, including its corresponding current construction action segment identifier, action deviation event identifier, and generated absolute timestamp.

[0013] As a further aspect of the present invention, the step of sorting and logically connecting multiple supervision detail fragments generated within a continuous time window according to the time constraint to form a complete supervision detail document for the target construction area specifically involves: Set a configurable monitoring period as the continuous time window, and collect all supervision detail fragments generated for the target construction area within the continuous time window; Based on the absolute timestamp in the metadata attached to each supervisory rule fragment, all supervisory rule fragments are sorted in ascending order by time. Based on the current construction action segment identifier corresponding to each supervision detail segment, multiple supervision detail segments belonging to the same typical construction activity sequence are grouped into the same group; Within each group, based on the process time constraints and logical relationships defined in the construction procedure knowledge graph, check whether there is a causal or progressive relationship between the action deviation events described in adjacent supervision detail segments; If a causal or progressive relationship exists, then when generating the document, insert connecting words and transitional sentences indicating the causal or progressive relationship between the text of the two supervisory detail fragments; Following the chronological order and grouping relationships, all the supervision details fragments and their connecting statements are sequentially assembled, and a document title, generation time, and target construction area overview section are added, ultimately compiling them into a structured and complete supervision details document.

[0014] As a further aspect of the present invention, it also includes: Establish a hyperlink mapping relationship between the complete supervision details document and the three-dimensional real-scene model of the target construction area; In the three-dimensional real-world model, the spatial locations corresponding to the spatiotemporal coordinates of the action deviation events described in the supervision guidelines document are highlighted. When a user clicks on a highlighted mark in the 3D real-world model, the system automatically locates and displays the corresponding supervision details and related supervision items in the complete supervision details document. The establishment of the hyperlink mapping relationship between the complete supervision details document and the 3D reality model of the target construction area is specifically as follows: A three-dimensional real-scene model of the target construction area is obtained by oblique photography or laser scanning; The spatiotemporal coordinates of the action deviation events associated with each text segment of the complete supervision rules document are analyzed, and the spatiotemporal coordinates include position information in the geodetic coordinate system. Establish a coordinate transformation relationship between the coordinate system of the three-dimensional reality model and the geodetic coordinate system; Using the coordinate transformation relationship, the spatiotemporal coordinates of each of the action deviation events are converted into the three-dimensional model coordinates in the three-dimensional real scene model; In the database of the three-dimensional real-scene model, a virtual anchor point is created for each coordinate of the three-dimensional model, and an index association is established between the virtual anchor point and the unique identifier of the corresponding supervision detail text fragment in the complete supervision detail document.

[0015] As a further aspect of the present invention, the present invention also includes an automatic generation system for scenario supervision details based on big data processing. The system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, it implements the steps of the above-described automatic generation method for scenario supervision details based on big data processing.

[0016] Compared with the prior art, the advantages and positive effects of the present invention are as follows: A construction procedure knowledge graph incorporating temporal and spatial constraints is constructed, transforming the clauses in the standard text and design drawings into a structured knowledge model that can be processed by computer programs. This model can automatically perform compliance matching for construction actions perceived on-site, determining whether actions violate specific constraints such as work sequence, time requirements, or safety distances through reasoning. This transforms compliance judgment from a model based on simple rules or human experience to automated reasoning based on explicit, structured procedure knowledge. The system can accurately output the specific type of deviation event and its corresponding spatiotemporal location information, and automatically trace and associate it with relevant specific supervision clauses and safety regulations based on the preset relationships in the knowledge model, thereby establishing a direct and traceable link from specific on-site actions to abstract regulatory clauses.

[0017] By integrating specific deviation parameters with associated regulatory clauses through parameterized templates, structured supervision detail fragments are dynamically generated. These fragments are then automatically sorted and linked into a complete supervision detail document based on chronological order and process logic. For each identified deviation, the process generates a standard format fragment containing a problem description, spatiotemporal coordinates, violation justification, and action instructions. Previously discrete and disordered alarm or event records are automatically organized and arranged into a logically coherent and comprehensive supervision guidance document based on the inherent timeline and process dependencies of the construction process. This transforms the supervision output from scattered information points into a sequenced set of instructions with a clear timeline and operational logic, directly applicable to guide on-site inspections and corrective actions. This achieves automated conversion and structured output of monitoring information into management action instructions. Attached Figure Description

[0018] Figure 1This is a flowchart of the method for automatically generating scenario supervision rules based on big data processing as described in this invention; Figure 2 This is a flowchart for multimodal perception data fusion and action segment extraction. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] See Figure 1 The overall implementation scheme of the automatic generation method for scenario supervision details based on big data processing is as follows: The method begins by acquiring historical engineering archive data and real-time collected on-site multimodal perception data of the target construction area. The historical engineering archive data is destructured to extract typical construction activity sequences and associated environmental status tags. The real-time collected multimodal perception data undergoes spatiotemporal alignment and fusion processing to generate a fused perception data stream, from which current construction action segments matching typical construction activity sequences are extracted. Based on the project specification text and design drawing data separated from the historical archives, a construction procedure knowledge graph containing time and space constraints is constructed. Using this construction procedure knowledge graph, compliance matching analysis is performed on the current construction action segment to identify deviation events from standard procedures and their spatiotemporal coordinates. Based on the type and spatiotemporal coordinates of the action deviation events, they are back-linked to relevant supervision items and safety clauses in the construction procedure knowledge graph. Subsequently, these supervision items, safety clauses, and specific parameters of the action deviation events are integrated to dynamically assemble and generate a supervision detail segment for the current construction action segment. Finally, the multiple supervision detail fragments generated within the continuous time window are sorted and logically connected according to time constraints to form a complete supervision detail document for the target construction area.

[0022] In one embodiment of the present invention, the initial step is to deconstruct historical engineering archive data to extract typical construction activity sequences and corresponding environmental status labels, separating project specification text, design drawing data, and past construction logs. Extracting typical construction activity sequences and corresponding environmental status labels from past construction logs involves a series of data processing operations. Natural language processing technology is used to parse the historical engineering archive data, identifying and separating text-formatted project specification text, image-formatted design drawing data, and structured past construction logs. Event extraction is performed on the unstructured text descriptions in the past construction logs to identify each specific construction operation recorded in the logs, including the operation execution time, the personnel performing the operation, and the associated machinery and materials. The identified specific construction operations are arranged according to their execution time to construct an initial construction activity sequence. Pattern mining is performed on the initial construction activity sequence to identify construction activity sub-sequences that recur in multiple historical projects and have fixed operation steps. These construction activity sub-sequences are marked as typical construction activity sequences. In some embodiments, the pattern mining process uses frequency analysis to identify typical construction activity sequences, defining a typicality measurement formula:

[0023] in: This represents a subsequence of construction activities. Indicates the first The initial construction activity sequence from the past construction logs of a historical project. It is an indicator function, when the subsequence It is a sequence The function value is 1 if the subsequence is a given value, otherwise it is 0. This is the total number of historical projects. Calculate subsequence The proportion appearing in historical projects. When When the value exceeds the preset threshold, the construction activity subsequence It is marked as a typical construction activity sequence. The temperature, humidity, light intensity, and wind speed data recorded simultaneously with each operation step in the typical construction activity sequence are extracted from past construction logs, and the temperature, humidity, light intensity, and wind speed data are used as environmental status labels and bound to the corresponding operation steps.

[0024] In specific implementation, a data template is established for each typical construction activity sequence. The data template includes a list of operational steps for the typical construction activity sequence and the value range of environmental status labels bound to each operational step. The value range of the environmental status labels is determined by statistically analyzing the minimum and maximum values ​​of environmental parameters associated with each operational step in historical data. Optionally, event extraction techniques can employ rule-based methods or machine learning models, such as using a pre-trained named entity recognition model to extract construction operation entities and attributes. It is understood that the refinement of typical construction activity sequences relies on the completeness and consistency of historical engineering archive data, and data preprocessing steps such as cleaning and standardization help improve the accuracy of typical construction activity sequences. In some embodiments, the binding process of environmental status labels involves time alignment, matching the execution timestamps of operational steps with the timestamps of environmental data recorded by sensors to ensure correct association. Optionally, for cases where environmental data is missing, data interpolation from nearby time points or estimates based on historical patterns can be used to fill in the gaps. It is understood that the establishment of the data template provides a structured reference for subsequent matching of real-time sensing data, and the list of operational steps for the typical construction activity sequence and the value range of environmental status labels together define the expected pattern of standard construction activities.

[0025] In one embodiment of the present invention, see [reference] Figure 2 In practical implementation, real-time acquired multimodal sensing data is spatiotemporally aligned and fused to generate a fused sensing data stream. This involves receiving real-time sensing data from multiple types of sensors deployed in the target construction area. The sensing data includes video streams, audio streams, depth images, and IoT sensor readings. Each type of sensing data carries a timestamp and the location code of the acquisition device. A unified spatiotemporal coordinate system is established, and all types of sensing data are mapped to this system based on timestamps and location codes to achieve spatiotemporal alignment of multi-source data. Within this unified spatiotemporal coordinate system, feature-level fusion is performed on data from different sensors at the same spatiotemporal point. The visual features of the video stream, the acoustic features of the audio stream, the spatial features of the depth image, and the numerical features of the IoT sensor readings are standardized to eliminate the influence of the original physical dimensions. In some embodiments, the standardization process employs Z-score normalization, defining a feature vector for a specific mode. The standardized formula is:

[0026] in: This represents the mean vector of the feature vector over the historical training dataset. The standard deviation vector of the feature vector over the historical training dataset represents the standardized feature vector. These are dimensionless numerical vectors. The standardized modal feature vectors are weighted and concatenated to form a multidimensional feature vector, defining the fused sensing data stream at spatiotemporal points. Eigenvectors at:

[0027] in: Represents the standardized visual feature vector. This represents the standardized acoustic feature vector. Represents the standardized spatial geometric feature vector. Represents the standardized numerical feature vector of an IoT sensor, scalar coefficients These are the fusion weight coefficients for each modal feature, whose values ​​are pre-configured based on sensor accuracy and task importance, and satisfy normalization conditions. The fused sensing data stream is input into a pre-trained action recognition model. Using data templates of typical construction activity sequences as a reference, the model identifies and segments data segments in the fused sensing data stream that match the action patterns of these sequences in real time. Each identified data segment is labeled with its corresponding typical construction activity sequence identifier, and the start and end times, as well as the spatial range it covers, are recorded, thus generating the current construction action segment. In practical implementation, a construction procedure knowledge graph containing time and spatial constraints is constructed based on project specification texts and design drawing data. Deep semantic analysis is performed on the project specification texts to identify all key entities and their attributes related to construction procedures, quality requirements, safety standards, and acceptance conditions, as well as the hierarchical, dependency, and causal relationships between these key entities. The design drawing data is parsed to extract the two-dimensional planar layout, three-dimensional structural model, component size information, and key coordinate points of the construction area. The key entities and their relationships obtained from semantic analysis are mapped with the spatial layout and structural information extracted from the design drawings, assigning each key entity its corresponding location range and geometric constraints in the actual construction space. Building upon the association mapping, a time dimension is introduced. The preset start time, end time, duration, and temporal logical relationships between each construction procedure are parsed from the project specification text to form a time constraint network. Optionally, the temporal logical relationships include "complete-start," "start-start," and "complete-complete" relationships, represented in the time constraint network using directed edges and time delay parameters. Key entities, entity attributes, semantic relationships between entities, spatial constraint information, and the time constraint network are integrated and stored and represented using a graph structure. Nodes represent entities or attributes, and edges represent relationships, spatial constraints, or temporal constraints, thus constructing a construction procedure knowledge graph. In the construction procedure knowledge graph, each edge representing a procedure is assigned a state attribute to identify the compliance status, progress status, or alarm status of the represented procedure during real-time monitoring.

[0028] In some embodiments, deep semantic analysis employs a domain ontology-based information extraction method. Key entities in the project specification text, such as "concrete pouring" and "reinforcement binding," are identified and linked to a predefined construction procedure ontology library. It can be understood that the construction procedure knowledge graph achieves a close integration of structured procedure knowledge with spatial and temporal information, with spatial constraint information derived from precise geometric coordinates and dimensions obtained through the parsing of design drawing data. Optionally, the knowledge graph can be stored using a graph database, facilitating complex graph traversal and relational query operations. It can be understood that setting state attributes for procedure edges enables the knowledge graph to dynamically reflect the actual progress and compliance status of on-site construction activities, with the values ​​of the state attributes being updated in real time during subsequent compliance matching analysis.

[0029] In one embodiment of the present invention, a construction procedure knowledge graph is used to perform compliance matching analysis on the current construction action segment to identify deviation events from the standard procedure and their spatiotemporal coordinates. The specific action type, action execution subject, action target, and actual time and spatial coordinates of the action are parsed from the current construction action segment. Using the action execution subject and action target as query conditions, a search is performed in the construction procedure knowledge graph to locate the standard procedure node that the action execution subject should perform on the action target, as specified in the graph. The attribute information associated with the standard procedure node is read, including the description of the standard procedure's operation steps, the allowed standard time window, the allowed standard spatial area, the required environmental conditions, and the required tools or materials. The actual time, actual spatial coordinates, and real-time environmental condition data extracted from the fused sensing data stream are compared one by one with the standard time window, standard spatial area, and required environmental conditions associated with the standard procedure node. In some embodiments, a deviation indication function is defined for determining time deviations.

[0030] in: Indicates the actual timestamp of the action. Indicates the start timestamp of the standard time window. Indicates the end timestamp of the standard time window, when or When the function value is 1, it indicates that a time deviation has occurred; otherwise, it is 0. When the actual time exceeds the standard time window, or the actual spatial coordinates exceed the standard spatial area, or the real-time environmental status data does not meet the required environmental status conditions, an action deviation event is determined to have occurred. The type of action deviation event is recorded as time deviation, spatial deviation, or environmental condition deviation, and the actual time and actual spatial coordinates corresponding to the action deviation event are recorded as spatiotemporal coordinates.

[0031] In practical implementation, based on the type and spatiotemporal coordinates of the action deviation event, the relevant supervision entries and safety clauses are traced back in the construction procedure knowledge graph. The type of action deviation event determines the adjacent process nodes or related entities affected by the type of deviation in the construction procedure knowledge graph. Starting from the process node where the action deviation event occurred, backward tracing is performed along causal or dependency edges in the construction procedure knowledge graph to find the superior process node or prerequisite node that caused the action deviation. Simultaneously, a search is performed along hierarchical edges in the construction procedure knowledge graph to locate the superior supervision category node covering the process node. Specific supervision entries are attached to the superior supervision category node. Optionally, a graph traversal algorithm is used for backward tracing, starting from the current node and searching upstream nodes in the opposite direction of the "cause-to-effect" relationship or the "dependency" relationship. Based on the specific parameters of the action deviation event, safety standard entities related to the specific parameters are retrieved in the construction procedure knowledge graph, and the detailed safety clause text linked to the safety standard entities is extracted. It is understood that specific parameters include, but are not limited to, the type of the entity performing the action, the material properties of the object affected by the action, or the wind speed value at the time of the action. These parameters are used as keywords for matching queries in the safety standard entity index of the knowledge graph. The traced parent process node or prerequisite node, the retrieved supervision entries, and the extracted safety clause text are combined as a set of rules related to the action deviation event for output.

[0032] In some embodiments, spatial deviation is determined by calculating the minimum Euclidean distance between the actual spatial coordinates and the boundary of the standard spatial area. When the minimum distance is greater than zero, the coordinate point is determined to be outside the area. It is understood that the satisfaction judgment of environmental conditions involves comparing real-time environmental condition data with the threshold range required by the standard process node. For example, if the real-time humidity data is lower than the standard humidity lower limit, an environmental condition deviation is triggered. Optionally, the output format of the rule basis set is structured, including a list of traceability path nodes, a set of supervision item identifiers, and safety clause text blocks. This information provides direct text and rule material for the subsequent generation of supervision detail fragments.

[0033] In one embodiment of the present invention, specific parameters of supervision items, safety clauses, and action deviation events are integrated to dynamically assemble and generate a supervision detail fragment for the current construction action segment. A supervision detail fragment template library is established, containing text generation templates for different types of action deviation events. Each template includes fixed statement sections and variable parameter placeholders. Based on the type of action deviation event, a corresponding text generation template is selected from the template library. The specific parameters of the action deviation event, including deviation type, occurrence time, occurrence location, and involved subjects and objects, are filled into the corresponding variable parameter placeholders of the selected text generation template. The titles and key requirement statements of the relevant supervision items and the core provisions of the safety clauses are inserted into the designated positions of the text generation templates in logical order. In some embodiments, the text generation template filling operation uses a string replacement algorithm, defining a replacement function:

[0034] in: The string representing the text generation template. Representing a mapping dictionary containing parameter names and values, the function iterates through all placeholder markers in the template and replaces them with the mapping dictionary. The corresponding parameter value string. Natural language generation technology is used to refine the sentences and improve the logical coherence of the text template after filling and inserting content, generating grammatically correct and clearly expressed supervision detail text fragments. Metadata is added to the generated supervision detail text fragments, including the corresponding current construction action fragment identifier, action deviation event identifier, and absolute timestamp of generation. Optionally, the supervision detail fragment template library is organized and managed using tables, which record template identifiers, action deviation event types, fixed sentence structures, and parameter placeholder lists. See Table 1.

[0035] Table 1: Structure of the Supervisory Rules Fragment Template Library

[0036] In practice, multiple supervision detail fragments generated within a continuous time window are sorted and logically connected according to time constraints to form a complete supervision detail document for the target construction area. A configurable monitoring period is set as the continuous time window, and all supervision detail fragments generated for the target construction area within the continuous time window are collected. All supervision detail fragments are sorted in ascending order of time based on the absolute timestamp in the metadata attached to each fragment. Multiple supervision detail fragments belonging to the same typical construction activity sequence are grouped into the same group based on the current construction action fragment identifier corresponding to each fragment. Within each group, based on the process time constraints and logical relationships defined in the construction procedure knowledge graph, it is checked whether there is a causal or progressive relationship between the action deviation events described in adjacent supervision detail fragments. If a causal or progressive relationship exists, connecting words and transition sentences indicating the causal or progressive relationship are inserted between the text of the two supervision detail fragments when generating the document.

[0037] All supervision detail fragments and their connecting statements are sequentially assembled according to chronological order and grouping relationships. A document title, generation time, and target construction area overview are added to ultimately compile a structured and complete supervision detail document. In some embodiments, causal relationship checks are performed by querying the dependency edges between process nodes corresponding to action deviation events in the construction specification knowledge graph, while progression relationship checks are based on the chronological order of action deviation events and the logical sequence of processes. It is understood that the monitoring cycle configuration is dynamically adjusted according to the construction project stage; a shorter interval is set for the foundation construction stage, and a longer interval is set for the main structure construction stage. Optionally, the output format of the complete supervision detail document adopts a standard document structure, including a cover, table of contents, main text, and appendix. The main text presents the supervision detail fragments in chronological order and grouping. It is understood that when adding the target construction area overview section, basic information about the area, such as project name, construction stage, and a list of key components, is extracted from the construction specification knowledge graph for population.

[0038] In one embodiment of the present invention, the specific implementation further includes the step of establishing a hyperlink mapping relationship between the complete supervision detail document and the three-dimensional real-scene model of the target construction area. This involves obtaining the three-dimensional real-scene model of the target construction area established through oblique photography or laser scanning, and parsing the spatiotemporal coordinates of the action deviation events associated with each supervision detail text fragment in the complete supervision detail document. The spatiotemporal coordinates include location information in the geodetic coordinate system. A coordinate transformation relationship is established between the coordinate system of the three-dimensional real-scene model and the geodetic coordinate system. This coordinate transformation relationship is used to convert the spatiotemporal coordinates of each action deviation event into three-dimensional model coordinates in the three-dimensional real-scene model. A virtual anchor point is created for each three-dimensional model coordinate in the database of the three-dimensional real-scene model, and an index association is established between the virtual anchor point and the unique identifier of the corresponding supervision detail text fragment in the complete supervision detail document. The spatial location corresponding to the spatiotemporal coordinates of the action deviation events described in the supervision detail document is highlighted in the three-dimensional real-scene model. In some embodiments, the coordinate transformation relationship is implemented through a seven-parameter transformation model, defining the transformation formula as follows:

[0039] in: This represents the coordinate vector of the action deviation event in the geodetic coordinate system. This represents a 3x3 rotation and scaling matrix. Represents the translation vector. This represents the calculated coordinate vector of the 3D model in the 3D reality model coordinate system. When the user clicks on the highlighted mark in the 3D reality model, the system automatically locates and displays the corresponding supervision detail text and related supervision items from the complete supervision detail document. The display process is achieved by querying the index association between the virtual anchor point and the unique identifier of the supervision detail text.

[0040] In practice, the highlighted markers are visually represented as semi-transparent colored cubes or glowing dots, suspended at their corresponding 3D model coordinates within the 3D reality model. The color of the highlighted markers is differentiated based on the type of action deviation event; for example, time deviations are marked in yellow, spatial deviations in red, and environmental condition deviations in blue. Optionally, the creation of virtual anchor points includes not only 3D model coordinate information but also attribute labels for the action deviation event type and summary information of the associated supervision detail text. This information is presented as a tooltip when the user hovers over the highlighted marker. It is understood that establishing coordinate transformation relationships requires parameter calculation based on the correspondence between the coordinate system used in constructing the 3D reality model and the geodetic coordinate system used in construction surveying. The calculation process uses two sets of coordinate values ​​from ground control points. In some embodiments, the index association is stored in a mapping table in a relational database. The mapping table fields include the virtual anchor point number, 3D model coordinates, a unique identifier for the supervision detail text, and a timestamp. Understandably, the display action triggered by a user clicking the highlighted marker is typically presented in the user interface as a sidebar or pop-up window. Simultaneously, the sidebar or pop-up window loads and highlights the corresponding supervisory detail text from the complete supervisory detail document, while also listing the relevant supervisory item titles. Optionally, the interactive rendering of the 3D reality model is implemented using a browser-side engine supporting WebGL technology. The highlighted marker, as an independent interactive 3D object, is loaded into the scene graph and maintains real-time communication with the backend mapping database.

[0041] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for automatically generating scenario-based supervision rules based on big data processing, characterized in that: The method includes: Acquire historical engineering archive data and real-time collected on-site multimodal perception data of the target construction area; The historical engineering archive data is destructured and extracted to identify typical construction activity sequences and corresponding environmental status labels. The real-time collected on-site multimodal sensing data is spatiotemporally aligned and fused to generate a fused sensing data stream, and the current construction action segment that matches the typical construction activity sequence is extracted from the fused sensing data stream. Based on the project specification text and the design drawing data, a construction procedure knowledge graph containing time constraints and space constraints is constructed. The construction procedure knowledge graph is used to perform compliance matching analysis on the current construction action segment to identify action deviation events that deviate from the standard procedure and their spatiotemporal coordinates. Based on the type and spatiotemporal coordinates of the aforementioned action deviation events, the relevant supervision items and safety clauses are traced back and associated in the construction procedure knowledge graph. By integrating the specific parameters of the supervision items, safety clauses, and action deviation events, a supervision detail fragment for the current construction action segment is dynamically assembled and generated; Multiple supervision detail fragments generated within a continuous time window are sorted and logically connected according to the time constraints to form a complete supervision detail document for the target construction area.

2. The method for automatically generating scenario supervision rules based on big data processing according to claim 1, characterized in that, The process of structuring and deconstructing the historical engineering archive data to extract typical construction activity sequences and corresponding environmental status labels involves the following steps: The project specification text, design drawing data and past construction logs are separated, and typical construction activity sequences and corresponding environmental status labels are extracted based on the past construction logs. Natural language processing technology is used to parse the historical project archive data, identify and separate the project specification text in text format, the design drawing data in image format, and the structured past construction logs; Event extraction is performed on the unstructured text descriptions in the past construction logs to identify each specific construction operation, operation execution time, operation personnel, and associated machinery and materials recorded in the logs; The identified specific construction operations are arranged in chronological order of their execution time to form an initial construction activity sequence; Pattern mining is performed on the initial construction activity sequence to identify construction activity sub-sequences that repeat in multiple historical projects and have fixed operation steps, and these construction activity sub-sequences are marked as typical construction activity sequences. Extract temperature, humidity, light, and wind speed data recorded simultaneously with each operation step in the typical construction activity sequence from the past construction logs, and bind the temperature, humidity, light, and wind speed data as environmental status tags to the corresponding operation steps; A data template is established for each of the typical construction activity sequences. The data template includes a list of operation steps for the typical construction activity sequence and a range of values ​​for the environmental status label bound to each operation step.

3. The method for automatically generating scenario supervision rules based on big data processing according to claim 2, characterized in that, The process involves spatiotemporal alignment and fusion of the real-time acquired on-site multimodal sensing data to generate a fused sensing data stream, and extracting the current construction action segment that matches the typical construction activity sequence from the fused sensing data stream. Specifically: The system receives real-time sensing data from various types of sensors deployed in the target construction area. The sensing data includes video streams, audio streams, depth images, and IoT sensor readings. Each type of sensing data is accompanied by a timestamp and the location code of the acquisition device. A unified spatiotemporal coordinate system is established, and all categories of the perceived data are mapped to the unified spatiotemporal coordinate system based on the timestamp and the location code to achieve spatiotemporal alignment of multi-source data. In the unified spatiotemporal coordinate system, feature-level fusion is performed on data from different sensors at the same spatiotemporal point. The visual features of the video stream, the acoustic features of the audio stream, the spatial features of the depth image, and the numerical features of the IoT sensor readings are combined into a multi-dimensional feature vector to form the fused sensing data stream. The fused sensing data stream is input into a pre-trained action recognition model. The action recognition model uses the data template of the typical construction activity sequence as a reference to identify and cut out data segments in the fused sensing data stream that match the action patterns of the typical construction activity sequence in real time. Each identified data segment is labeled with the identifier of its corresponding typical construction activity sequence, and the start time, end time, and spatial range covered by the data segment are recorded, thereby generating the current construction action segment.

4. The method for automatically generating scenario supervision rules based on big data processing according to claim 3, characterized in that, The construction of a construction procedure knowledge graph, which includes time and space constraints, based on the project specification text and the design drawing data, specifically involves: Deep semantic analysis was performed on the project specification text to identify all key entities and their attributes related to construction procedures, quality requirements, safety standards, and acceptance conditions, as well as the hierarchical, dependency, and causal relationships among these key entities. The design drawing data is parsed to extract the two-dimensional plan layout, three-dimensional structural model, component size information, and key coordinate points of the construction area; The key entities and their relationships obtained from semantic analysis are associated and mapped with the spatial layout and structural information extracted from the design drawings, and each key entity is assigned its corresponding location range and geometric constraints in the actual construction space. Based on the aforementioned association mapping, a time dimension is introduced to parse the preset start time, end time, duration, and temporal logical relationship between each construction procedure from the project specification text, forming a time constraint network. The key entities, entity attributes, semantic relationships between entities, spatial constraint information, and temporal constraint network are integrated and stored and represented using a graph structure. Nodes represent entities or attributes, and edges represent relationships, spatial constraints, or temporal constraints, thereby constructing the construction procedure knowledge graph. In the construction procedure knowledge graph, a status attribute is set for each edge representing a process, which is used to identify the compliance status, progress status, or alarm status of the representative process during real-time monitoring.

5. The method for automatically generating scenario supervision rules based on big data processing according to claim 4, characterized in that, The process of using the construction procedure knowledge graph to perform compliance matching analysis on the current construction action segment, identifying action deviation events that deviate from the standard procedure and their spatiotemporal coordinates, specifically involves: The specific action type, the subject of the action, the object of the action, and the actual time and spatial coordinates of the action are extracted from the current construction action segment. Using the subject performing the action and the object acting on the action as query conditions, a search is performed in the construction procedure knowledge graph to locate the standard process node that the subject performing the action should perform on the object acting on the action as specified in the graph. Read the attribute information associated with the standard process node, including the description of the standard process operation steps, the standard time window allowed to be executed, the standard spatial area allowed to be executed, the required environmental conditions, and the required tools or materials; The actual time, actual spatial coordinates, and real-time environmental status data extracted from the fused perception data stream are parsed from the current construction action segment and compared one by one with the standard time window, standard spatial region, and required environmental status conditions associated with the standard process node. When the actual time exceeds the standard time window, or the actual spatial coordinates exceed the standard spatial area, or the real-time environmental status data does not meet the required environmental status conditions, an action deviation event is determined to have occurred. The type of the action deviation event is recorded as time deviation, spatial deviation, or environmental condition deviation, and the actual time and actual spatial coordinates corresponding to the action deviation event are recorded as the spatiotemporal coordinates.

6. The method for automatically generating scenario supervision rules based on big data processing according to claim 5, characterized in that, The step of backtracking and associating the action deviation event with relevant supervision items and safety clauses in the construction procedure knowledge graph based on the type and spatiotemporal coordinates is as follows: Based on the type of the action deviation event, determine the adjacent process nodes or related entities affected by the deviation type in the construction procedure knowledge graph; Starting from the process node where the action deviation event occurs, trace back along the causal or dependency relationship edges in the construction procedure knowledge graph to find the superior process node or prerequisite node that caused the action deviation. Simultaneously, by searching along the hierarchical relationship edge in the construction procedure knowledge graph, the superior supervision classification node covering the process node is located, and the superior supervision classification node has specific supervision items attached to it. Based on the specific parameters of the action deviation event, retrieve the safety standard entity related to the specific parameters in the construction procedure knowledge graph, and extract the detailed safety clause text linked to the safety standard entity. The traced superior process node or prerequisite node, the retrieved supervision item, and the extracted safety clause text are collectively output as a set of rules related to the action deviation event.

7. The method for automatically generating scenario supervision rules based on big data processing according to claim 6, characterized in that, The process of integrating the specific parameters of the supervision items, safety clauses, and action deviation events to dynamically assemble and generate a supervision detail segment for the current construction action segment is as follows: Establish a supervision rule fragment template library, which contains text generation templates for different types of action deviation events. Each template contains a fixed statement section and variable parameter placeholders. Based on the type of the action deviation event, a corresponding text generation template is selected from the template library; The specific parameters of the action deviation event, including deviation type, occurrence time, occurrence location, and the subject and object involved, are filled into the corresponding variable parameter placeholders of the selected text generation template. Insert the titles and key requirements of the relevant supervision items, as well as the core provisions of the security clauses, into the designated positions in the text generation template in logical order; Using natural language generation technology, the text template after filling and inserting content is processed to polish sentences and ensure logical coherence, generating grammatically correct and clearly expressed supervisory detail text fragments. Add metadata to the generated supervision detail text, including its corresponding current construction action segment identifier, action deviation event identifier, and generated absolute timestamp.

8. The method for automatically generating scenario supervision rules based on big data processing according to claim 7, characterized in that, The process of sorting and logically connecting multiple supervision detail fragments generated within a continuous time window according to the time constraint to form a complete supervision detail document for the target construction area specifically involves: Set a configurable monitoring period as the continuous time window, and collect all supervision detail fragments generated for the target construction area within the continuous time window; Based on the absolute timestamp in the metadata attached to each supervisory rule fragment, all supervisory rule fragments are sorted in ascending order by time. Based on the current construction action segment identifier corresponding to each supervision detail segment, multiple supervision detail segments belonging to the same typical construction activity sequence are grouped into the same group; Within each group, based on the process time constraints and logical relationships defined in the construction procedure knowledge graph, check whether there is a causal or progressive relationship between the action deviation events described in adjacent supervision detail segments; If a causal or progressive relationship exists, then when generating the document, insert connecting words and transitional sentences indicating the causal or progressive relationship between the text of the two supervisory detail fragments; Following the chronological order and grouping relationships, all the supervision details fragments and their connecting statements are sequentially assembled, and a document title, generation time, and target construction area overview section are added, ultimately compiling them into a structured and complete supervision details document.

9. The method for automatically generating scenario supervision rules based on big data processing according to claim 1, characterized in that, Also includes: Establish a hyperlink mapping relationship between the complete supervision details document and the three-dimensional real-scene model of the target construction area; In the three-dimensional real-world model, the spatial locations corresponding to the spatiotemporal coordinates of the action deviation events described in the supervision guidelines document are highlighted. When a user clicks on a highlighted mark in the 3D real-world model, the system automatically locates and displays the corresponding supervision details and related supervision items in the complete supervision details document. The establishment of the hyperlink mapping relationship between the complete supervision details document and the 3D reality model of the target construction area is specifically as follows: A three-dimensional real-scene model of the target construction area is obtained by oblique photography or laser scanning; The spatiotemporal coordinates of the action deviation events associated with each text segment of the complete supervision rules document are analyzed, and the spatiotemporal coordinates include position information in the geodetic coordinate system. Establish a coordinate transformation relationship between the coordinate system of the three-dimensional reality model and the geodetic coordinate system; Using the coordinate transformation relationship, the spatiotemporal coordinates of each of the action deviation events are converted into the three-dimensional model coordinates in the three-dimensional real scene model; In the database of the three-dimensional real-scene model, a virtual anchor point is created for each coordinate of the three-dimensional model, and an index association is established between the virtual anchor point and the unique identifier of the corresponding supervision detail text fragment in the complete supervision detail document.

10. A system for automatically generating scenario supervision rules based on big data processing, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the automatic generation method for scenario supervision details based on big data processing as described in any one of claims 1 to 9.

Images